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Preface 



These are the proceedings of the second symposium on Generative and Com- 
ponent-Based Software Engineering that was held in Erfurt, Germany, on Oc- 
tober 9-12, 2000, as part of the Net. Object Days conference. The GCSE sym- 
posium was born in 1999 at the Smalltalk and Java in Industry and Educa- 
tion Conference (STJA), the precursor to the Net. Object Days conference. The 
GCSE symposium grew out of a special track on generative programming that 
was organized by the working group “Generative and Component-Based Soft- 
ware Engineering” of the “Gesellschaft fiir Informatik” EG 2.1.9 at STJA in the 
two years 1997 and 1998. The GCSE symposium covers a wide range of related 
topics from domain analysis, software system family engineering, and software 
product lines, to extendible compilers and active libraries. 

The second GCSE symposium attracted 29 submissions from all over the 
world. This impressive number demonstrates the international interest in gen- 
erative programming and related fields. After a careful review by the program 
committee, 12 papers were selected for presentation. We are very grateful to 
the members of the program committee, all of them renowned experts, for their 
dedication in preparing thorough reviews of the submissions. 

Special thanks go to Elke Pulvermiiller, Andreas Speck, Kai Bollert, Detlef 
Streitferdt, and Dirk Heuzeroth, who continued the tradition from GCSE’99 and 
organized a special conference event, the Young Researchers Workshop (YRW). 
This workshop provided a unique opportunity for young scientists and Ph.D. 
students to present their ideas and visions of generative programming and related 
topics and to receive thorough critique and feedback from senior experts in the 
field. 

We are also indebted to the keynote speakers and tutorial presenters, Paul 
Bassett, Doug Smith, and Michel Tilman, for their contribution to GCSE 2000. 
Fortunately, we have papers from two of the three keynote speakers for these 
proceedings. Unfortunately, there is no paper for the keynote talk on “Software 
Development by Refinement” given by Doug Smith of the Kestrel Institute. Fi- 
nally, we wish to thank all who put in their efforts and helped to make this 
symposium happen, especially the authors and the Net. Object Days organizers. 

We hope you will enjoy reading the GCSE 2000 contributions, and invite you 
to contribute to future symposiums. 



October 2000 



Greg Butler 
Stan Jarzabek 
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The Theory and Practice of Adaptive 
Components 



Paul G. Bassett 

Netron, Inc., Canada 
pbassettOnetron.com 



Abstract. It is well known that modifying software by hand, especially 
someone else’s, is not only slow and tedious but so notoriously error- 
prone that we strive for components that never need changing — black- 
box building blocks. Unfortunately, we lack the omniscience required 
to engineer such parts, especially when our problem domains are ill- 
defined. A promising alternative is to make each component adaptive 
— a simple mark-up language (e.g., frame technology) converts each 
component’s properties (i.e., granularities that are meaningful within a 
given domain) into a default value that can be overridden or extended by 
other components in a rapid yet reliable manner. Adaptive components 
constitute a gray-box strategy — black when their properties fit the 
context as is, otherwise white. The approach works by automating the 
tedious and error-prone aspects of construction and modification, while 
localizing all the unique properties of an object or program in its root 
component. Each root exercises complete control over the structure and 
content of the code that its hierarchy of components emits. Such custom 
roots constitute only 5-15% of a typical system. In other words, 85-95% 
reuse is the norm. Along with independently gathered statistical evidence 
of efficacy in constructing and evolving business systems of up to 10 
million lines of code, the further implications of adaptive components 
for software engineering are presented. 



1 Introduction 

Good morning ladies and gentlemen. It gives me great pleasure to be able to talk 
to you about my favorite software engineering topic: adaptive components. In 
fact, I had trouble writing this address, as there are so many ways to approach 
them. I decided to introduce them by tracing their origins and current status. 
Having set the stage, we will see adaptive components as they have been realized 
in frame technology, discussing how they change the way we think about domain 
modeling and componentization. By then you will be wondering how well formal 
adaptivity works. I’ll present independently gathered statistics that show savings 
in time and effort that many find too good to be true! So what’s the catch? I’ll 
close with my thoughts on what it takes for adaptive components to find their 
way into general practice. 

G. Butler and S. Jarzabek (Eds.): GCSE 2000, LNCS 2177, pp. 1-14, 2001 . 
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“CaU/Return” mechanisms have been built into computer hardware for more 
than 40 years. Both macros and subroutines have been standard features of as- 
sembly languages for almost as long. Software engineers have been generating 
code and building systems from components almost since the inception of pro- 
gramming. 

In principle, generators, subroutines, and macros all have the potential to 
simplify program understanding and reduce coding effort. In practice, generators 
have had a checkered history, the reasons for which I shall address later. Let me 
now compare and contrast macros with subroutines. 

Curiously, macros fell into disuse as higher level languages emerged. Only 
a handful of 3GLs e.g., C, contained them, and even fewer 4GLs. Subroutines, 
on the other hand, flourished in more and more forms — function calls, over- 
loaded operators, coercion, recursion, reentrant, call-by-name, call-by-value, and 
object methods, to name a few. (For the sake of simplicity, I shall use the term 
subroutine to refer to all of the above.) 



2 Duality 

I’ve pondered long and hard about why subroutines succeeded while macros 
languished. 

One technical reason is that macros typically suffer from the so-called “global 
variable” problem. That is, macros can overwrite each other’s variables, making 
macros unpredictable and error-prone. Global variables also afflicted programs 
and subroutines, inspiring the evolution of block-structured programming lan- 
guages. Macro languages could easily scope their variables to nested blocks too, 
but for some reason most don’t. 

Another technical objection, at least in the early days, was that memories 
were small and expensive, causing people to avoid in-line macro expansions. 
Today that reason is totally obsolete. 

A more insidious reason is the myth that macros and subroutines are com- 
petitive ways to do much the same thing, and after all, don’t programmers 
already have too many ways hang themselves? In fact they are complementary, 
not competing mechanisms for implementing components. 

While it is true that both are parameter-driven, macros operate only at 
construction time to emit program structures, and subroutines operate only at 
run time to emit data. Macros operate at the component meta-level: components 
that construct components, including, of course, subroutines. 

It’s hard to overstate the significance of this fact. To give you one facet of 
its power, consider that one macro can give rise to an infinity of polymorphs, 
depending on its parameter inputs. So, rather than drowning in a sea of look- 
alike functions, systems can be architected, built, and evolved with small set of 
normalized basis functionals. 

The myth is not only false but it blinds us to a new universe of opportu- 
nity. What is that universe? In our business, when the word “architecture” is 
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heard, we automatically think of the structure and modularization of our run- 
time systems. The issues driving the architecture have to do with modularity, 
functionality, efficiency, and ease-of-use in distributed environments. But at the 
meta-level, there is a dual set of architectural questions having to do with min- 
imizing complexity while optimizing generality and adaptability. Given that a 
construction architecture must implement its run-time counterpart, the former 
is even more important to a well-engineered system than the latter. Yet hardly 
anyone pays attention to it! Why has the build/run duality, a source of so much 
insight and simplifying power, gone largely unrecognized? 



3 Softness 

Before proceeding, further into construction-time, we need to pause and ask 
ourselves a very basic question: Why do we solve problems using software at all? 
The case could be made that, especially with the advent of chip foundries, all 
could and should be burned into chips. Of course, a moment’s thought supplies 
the answer. The mison-d’etre of keeping our solutions in software is ease of 
modification. Our first version of a system often merely tells us that we have 
solved the wrong problem. Without software most of the systems that power 
modern society would be beyond our grasp simply because it’s totally infeasible 
to build special purpose machines for every version that such systems must evolve 
through. 

Given the fundamental importance of modifiability it is ironic that we rarely 
take the time to explicitly engineer our systems so that they conserve this prop- 
erty. As a result, systems once soft become brittle to the point that decrees are 
issued that no one is to modify them on pain of dismissal. Why have we ignored 
a meta-problem with such expensive consequences? 

Another meta-problem is the highly non-linear nature of the beast. Systems 
are said to behave linearly if the effect of a change is proportional to its size. But 
with software, change a single symbol and the result could range anywhere from 
totally benign to the crash of 200-million dollar Martian spacecrafts, or worse. 

Last but not least, billions of years of evolution have conditioned us to ex- 
pect everything to be made from concrete, stable, parts. We continue to design 
software systems as if they were cars and TV sets. Indeed, the 00 paradigm 
perpetuates the myth that the world can be built from immutable components 
— Lego blocks. 

Before I raise too many hackles, let me explain why I rail against this at- 
titude. Problem domains come in many flavors. When the domain is both well 
understood and stable, it seems natural to create cost-effective, efficient solutions 
in terms of fixed building blocks. Such domains are constrained by the laws of 
physics, laws that are well understood, true for all space and time, and unbreak- 
able. Statistics routines, embedded systems, and power-plant control systems are 
examples. In these domains, ease of modification is a non-issue and the software 
is often burned into chips. 
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But most interesting domains are not so well behaved. Typically, the require- 
ments are conflicting, poorly understood by the stakeholders who also do not 
share a common understanding of them, and they change with time, sometimes 
radically. These issues pertain to the much larger category of ill-defined domains. 
Business applications are certainly not constrained by the laws of physics; most 
of the software on the web, and a myriad of other domains too numerous to 
mention all exhibit engineering degrees of freedom not available in the physical 
world. 

As we explore and experiment with what works and what doesn’t, ill-defined 
domains evolve into better-defined ones. Commodity packages supercede once- 
custom software for those domains. We take them for granted and stand on their 
shoulders, so to speak, so that we can tackle new the ill-defined domains inspired 
by our wealth-creation fantasies. And so the cycle repeats. 

But packaged systems never completely stabilize. As one version succeeds 
another, and as product lines are created for different market niches, they need 
to be engineered for cost-effective change. 

None of this is new to anyone, so why have we done such a poor job of 
engineering our systems for perpetual ease-of change? We continue to believe 
software should be built from immutable building blocks because, I contend, 
evolution has conditioned us to see the world as being built that way. The multi- 
dimensional universe of software structures possesses inherently new engineering 
degrees of freedom, which we are naturally slower to grasp. 

Reinforcing this collective blind spot is the deep psychological need to believe 
that only we are capable of making the fussy changes that software needs. On 
the other hand, we are so poorly equipped to make them reliably that we’ve 
institutionalized the idea that because we can build anything from fixed parts, 
and therefore we should do so. 

I have a simple reductio ad absurdum rebuttal. Just two fixed parts are suffi- 
cient to build all software — 0 and 1. This is about as sensible as manufacturing 
cars from atoms. With atomic force microscopes we can attempt it, but should 
we? 

The engineering issue concerns how to model the “natural” graininess of any 
domain with corresponding parts. Here “natural” means allowing us to highlight 
the important properties and ignore the irrelevant ones. This strategy leads to 
a complexity-minimized vocabulary for thinking about and designing system 
architectures. When a domain is well defined, its stability permits (but does 
not require) flxed parts. To the extent that it is ill defined, the parts should be 
adaptable to unforeseen requirements, reliably and without undue increases in 
complexity. The good news is that adaptive components not only make this is 
possible, but they benefit well-defined domains too! 

4 Context Sensitivity 

To prepare for the meat of adaptive components, we need to discuss context 
freedom and context sensitivity. These are relative terms, spanning a spectrum 
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of possibilities. At run time, the smaller and simpler the assumptions that a 
component needs to make about its interface, the more contexts it will satisfy 
and hence fit into. Hence the more context-free and reusable it is. For context 
sensitive parts, the converse is true. To a first approximation, context freedom 
is a nice property to have. 

The Lego block metaphor implies that all building blocks should be context 
free. But there is a problem. When a collection of such parts come together 
in a specific context they typically clash because they are literally ignorant of 
each other. Especially when components originate from different contexts and 
authors, the probability quickly jumps to near certainty that they will make 
inconsistent assumptions about mundane things like data formats and units of 
measure. 

Is this failure to fit into a new context the fault of the context-free component? 
Not usually. Rather, it is the mutual interaction of the components in a given 
context that is the source of the difficulties, not the individual components. 
This is why subclassing in 00 is so rampant and why libraries of thousands of 
polymorphic classes arise, drowning us in seas of lookalikes. 

A variant of this strategy is to write additional “work around” code to undo, 
redo, and/or extend the existing functionality, but without modifying the original 
component or creating lookalikes. The disadvantages are several including: (a) 
more space (code), (b) more time (execution cycles), (c) more complexity (both 
in tortuous extra logic and in the fact that it is dislocated from the logic of the 
components with which it naturally belongs). 

On the other hand, we seldom modify the original component to suit the new 
context because we are likely to give ourselves enormous retrofit headaches. 

So our wish to believe that the world can be made out of Lego blocks puts 
us on the horns of a dilemma. 

There is an analogy that illuminates the real issue. Imagine you are in an 
auto plant watching cars being assembled. Suppose engineers came up with parts 
such as a fender that could be adapted to fit any make or model of car as each 
came down the line. Clearly this would cause a wonderful collapsing of the parts 
inventory and of the complexity of managing them all. But alas, it is a silly 
supposition; physical parts simply do not have the needed flexibility. But in 
software engineering, adaptive engineering degrees of freedom exist because the 
parts are non-physical. Such adaptive components are a very natural thing to 
have in software engineering, and the attendant benefits are large, as we shall 
see. 



Unfortunately, our current practices are still rooted in the physical paradigm, 
leading to the above dilemma. We make actual parts rather than meta-parts. 
By managing our much-reduced inventory of meta-parts we avoid the headaches 
and shrink software projects dramatically. Strong claims. I now explain how they 
work, and conclude with evidence of how well they work. 
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5 Adaptive Components Defined 

For any information domain, an adaptive component is a formal embodiment of 
an archetype together with rules for combining (sub)component-archetypes into 
context-specific structures and/or functions. — Webster’s Dictionary defines an 
archetype as “the original pattern or model from which all other things of the 
same kind are made.” 

I have characterized adaptive components in this way to emphasize that they 
have applications to any domain for which a symbolic language can be used to 
describe its contents. For example, manuals have been assembled from adap- 
tive components that explain how to use a software product that works a little 
differently, depending on which computer platform it is installed. The core func- 
tionality of the product is the same, but the user interface must adopt the look 
and feel of the appropriate operating system. So platform-specific components 
contain the information needed to adapt the core documentation components 
and each manual looks like it was written especially for its platform. While the 
statements to follow apply to this more general context, my focus will be the 
construction and evolution of software systems, because that is what I know 
best. 

The archetypes to which the definition alludes are the models of the domain’s 
entities and operations. The more generic archetypes can be thought of as generic 
roots of nouns and verbs in a domain-specific vocabulary. The more specific ones 
contain the modifiers — adjectives, adverbs, sentence agreement rules, prepo- 
sitions, etc. — that combine with the root concepts to form words, sentences, 
paragraphs, sections, chapters (e.g., executables), and documents (e.g. systems). 

Creating an archetype is not as hard as it may sound. Given a specific problem 
to solve, your solution can serve as your first cut at the archetype for all similar 
problems. You strive for a solution that typifies the others, then parameterize it 
to be adaptable and to adapt. Alternatives may turn out to be needed for a given 
class of problems, depending on the context. This is clearly an imperfect exercise. 
The good news, as we shall see, is that the facilities of adaptive components are 
forgiving of many errors of omission and commission. 

Technically, adaptive components are a kind of macros that “mark up” un- 
derlying texts. We are familiar with such markup languages as HTML and XML. 
XML in particular has the virtue of defining a standard meta-syntax indepen- 
dently of the actual syntax and semantics of the underlying text. It does this by 
embedding the text with a set of nested, labeled “brackets” . An XML-compliant 
sample might appear as: 

<patient> 

<name>Joe Blow</name> <drug-allergy>penicillin</drug-allergy> 
</patient> 

By parsing the embedded markups “browsers” can variously process bills, 
analyze patient medical records, update databases, or do whatever. The meta- 
notation is readable — it uses angle brackets, ampersands and semi-colons to 
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separate markup text from regular text. It structures the contents of any doc- 
ument as a tree, rendering it block structured, straightforward to parse, and to 
process. Many different processors can operate on the same text, say to display 
it in a graphical form (hiding the markup notation), or to compile it. 

I use frame technology to illustrate one of many ways to implement adap- 
tive components. For more information, see my book [1]. It consists of a frame 
(macro) processor (FP), operating on frame syntax, which was invented before 
XML but easily translated into it. As you would expect, each frame is one adap- 
tive component — a macro of marked up text. The text typically is a 3GL (e.g. 
COBOL, C) but could be a language to capture requirements, or a high level 
modeling language, or, as I’ve said, just about any text. 

The mark ups form nested tree structures, both within and among frames. 
— A frame library actually forms what in mathematics is called a semi-lattice, 
under the “adapt” partial ordering. FP processes each semi-lattice as if it were 
a tree of subassemblies — a component that has multiple parents is, in effect, 
cloned, one per parent. So with that understanding, I can refer to a frame hierar- 
chy or semi-lattice as a tree. — FP begins with a root frame, called a specification 
frame, and traverses its frame-tree in natural (depth first, left-to-right) order. 
By carrying out the mark-up rules, FP assembles one module of text per spec- 
ification frame. The emitted text should conform to the syntax of the marked 
up language, e.g., if the output is source-code, it should be compileable without 
errors. While the frames are strictly read only inputs to the FP, it is natural to 
pretend the frames are adapting each other, and in what follows that fiction is 
a useful shorthand. 

Frame trees stratify contexts, the specification frame being completely con- 
text specific (sensitive), and the leaf components being the most context-free. 
Each frame, being the root of a subtree, localizes the context information needed 
to achieve a seamless integration of the elements of that subtree. What follows 
outlines how frame technology implements this recursive idea. 



6 Handling Unpredictable and Unique Variability 

The main problem that adaptive components seek to solve is coping with ar- 
bitrary and unknowable change. Much research in domain analysis focuses on 
identifying the commonalities and the variabilities inherent in our domains prior 
to designing our systems. Adaptive component engineers take the position that 
much of this analysis is a waste of time. Why? Adaptivity makes it much easier 
and faster to assume that everything is potentially variable — it’s just that our 
ill defined domain has yet to reveal the specifics. Actually building a good first 
approximation is the fastest way to zero in on non-obvious variabilities. 

In spite of not being omniscient we can still complete our projects on time 
and within budget because adaptive components are engineered to tolerate our 
ignorance. Our building blocks can incorporate our improving understanding, 
usually without invalidating much of what went before. Sound like magic? Okay 
it’s time to show how such components work. (I apologize for having to introduce 
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some technical vocabulary. Hopefully, your reward for persevering will be a better 
insight into interesting engineering issues.) 

A frame’s overall structure is simple: 

<frame> name <body> frame-text </body> </frame>. 

Frame-text is a mixture of plain and marked-up text. Plain text encodes 
anything that is currently not known or anticipated to be variable, and FP 
emits it as-is. Let’s turn our attention to how the various mark-ups deal with 
variability. 

There are two kinds of parameter assignment. The first looks like this: 
<replace> name <by> expression </by> </replace> 

name is the name of a parameter but may itself be an expression. An ex- 
pression can be a combination of literals, <replace> parameter references, and 
arithmetic expressions that together evaluate to a literal text string. If, after 
evaluating its name, a given <replace> is allowed to alter the <replace> sym- 
bol table — which records the current assignments of parameters (symbols) and 
manages their scopes (see next paragraph) — then its expression is evaluated, 
and the result is entered into that table. 

<replace> is scoped by the following “global overrides local” rule: Values 
assigned to parameters inside frame F cannot be changed by any of F’s descen- 
dent frames. E.g., if F has set P to value V then V will replace all references to 
P within the subtree rooted in F. When F exits (i.e., FP finishes processing it), 
any parameters that F was able to <replace> become undefined. In effect, for F 
to set P, it must be undefined when F is invoked, and all other frames’ attempts 
to <replace> P are ignored until F exits. 

This scoping rule is not found in most languages and has several noteworthy 
implications for adaptive components. 

- Whenever context-free (local) default values fit the contexts of its more 
context-sensitive (global) ancestor frames, those parameters need not be 
overridden. In effect, each frame <replace>s the least number of param- 
eters sufficient to define its own context — i.e., ensure that itself and its 
component frames result in a seamless integrated whole. 

- Conversely, this is the “same as, except” principle in action. When you want 
to reuse a component subassembly, highlight the exceptions in the embedding 
context; the rest stays the same. 

- A frame is blissfully unaware that the contexts into which it is being embed- 
ded may override some of its parameters. This allows a context-free compo- 
nent to be evolved, rather than having to be created in mature form. How? 
Start with plain text that suits a single context — recall it gets emitted as-is; 
when a new context requires variability to elements of the plain text, simply 
replace those elements with parameter references whose default assignments 
are the original elements. Now the frame still works in all prior contexts, and 
new contexts can override as necessary. 
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- The recursive nature of nested frame trees enables each ancestor frame to 
seamlessly integrate ever larger subassemblies, a software-engineering basis 
for the truism, “the higher in the hierarchy, the more clout you have” . 

The way to utilize <replace> parameters is to embed references to them in 
frame-text using the syntax <; > name </;>, where name is an expression that 
evaluates to a symbol table entry. The mark-up is then replaced by the table’s 
corresponding value. When the underlying programming language is marked-up 
with <;> parameters, they cause data structures, interfaces, and logic to vary 
according to the embedding context. Just as importantly, as we shall see, such 
references also control the processing of frames. 

There is one other kind of assignment; it occurs when a frame adapts (invokes) 
a frame subassembly: 

<adapt [samelevel]> frame-name 

[<insert [before, after] > break-name 
<content> frame-text </content> 

</insert> ... ] 

</ adapt > 

frame-name is an expression that evaluates to the unique name of an ex- 
isting frame. Control transfers to the invoked frame and returns to this point 
after it and its subtree has been processed. Optional <insert>s form a list of 
parameters assignments that govern the subtree rooted in frame-name, not just 
the root frame. This ability of actual arguments to bind to formal arguments 
located anywhere in the subtree, together with “global overrides local” scoping, 
allows us to think of the entire subassembly as one seamless whole — which it 
will be, when the frame processor is finished. Being able to easily flip between 
integrated and discrete views of the structure simplifies the way frame engineers 
converge to the appropriate graininess of their domain models, the subject of 
several chapters in my book. 

<insert> causes frame-text to be inserted before, after, or instead of the 
matching <break>, which, as I’ve said, can be anywhere in the subtree. It looks 
like this: 

<break> name <default> frame-text </default> </break> 

Whereas <replace> evaluates its assigned expression during the assignment 
(in a possibly more global context), <insert> simply points to its unevaluated 
frame-text. Evaluation takes place upon reference, that is, after being “in- 
serted” into the <break> (of some less context-sensitive frame). In other words, 
<insert>s get evaluated in the local context while <replace>s may be evaluated 
in more global contexts. Note that when you want a <replace> to be evaluated 
in the local context, simply <insert> it. <insert>s work this way because the 
adapter frame plays the role of an editor, rewriting the adaptee. The <insert>ed 
frame-text often contains references to <replace> parameters whose values may 
have been set within the adaptee frame. 
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The basic assumption is that “the world is arbitrarily hairy” . If components 
are to be sufficiently adaptable, then there must be a way to cope with un- 
predictable and unique deviations from our archetypes. To do this, a <break> 
surrounds each structure within a frame, allowing for any element from the 
smallest paragraph to the entire frame to be “edited” . Clearly, such a powerful 
mechanism can be misused, but something like this is needed to handle arbitrary 
variability. Programmers give themselves this power, so why not harness it in a 
formal way? 

As with <replace>, frames <insert> only the exceptions needed to fit the 
context. And again, <break>s can be added later “for free” , forgiving our lack 
of omniscience. In ill-defined domains, you must get on with things well before 
you understand the best ways of engineering the components. 

Consider the reuse of components whose job is to define standard contexts. 
For example, suppose you have a library of adaptable components that work 
in both Sun and Mac operating environments (contexts). The natural way to 
isolate and reuse all the <replace> parameters needed to characterize the desired 
operating environment is to create one frame for each environment. Frame “S” 
contains all and only Sun’s parameter assignments; frame “M” does the same 
for Mac. But if I simply <adapt> S or M, the settings vanish (become undefined) 
as soon as the frame is processed. So, “<adapt samelevel> M” treats M as if it 
were inside the adapter frame, usually a specification frame. In such a case, the 
parameter assignments would govern the entire hierarchy. 

7 Handling Known Variants 

Whereas <insert> is designed for unique customizations, when a list of varia- 
tions is known, there should be a way to place them inside the frame where they 
would otherwise have to be <insert>ed. Then any adapter frame merely sets 
a <replace> parameter to select the appropriate variation. Thus, rather than 
having to <insert> the desired structure from some more context-sensitive level, 
the <select> permits the component to carry all the alternatives within itself. 

<select> <;>name</;> 

<when> relational-expression <content> frame-text </content> . . . 
[<otherwise> frame-text </otherwise>] 

</ select> 

<select> is a simple case statement, operating at construction time. The < ; > 
parameter’s value is tested against each relational (=, 7^, <, >, <, >) expression 
(including undefined) , and the true ones cause the associated frame-texts to be 
processed. An <otherwise> clause may be included to handle the case where no 
other selection is made. 

While simple, <select> has three important uses. We have already seen one 
— to eliminate redundancy where a list of variants defines the variability at 
a single point in the tree. The Mac versus Sun example hints at the converse 
issue — needing to have a single point for controlling variability at many places. 
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A global or non-functional property that must be implemented in a myriad of 
localized instantiations is one class of examples — say designing frames to handle 
multiple languages. <select> allows us to set a global parameter that causes a 
cascade of different effects to take place throughout the hierarchies. Each place 
where a variation is needed contains a <select> with the appropriate variations 
stored within, depending on the value of the global variables. 

The third important application of <select> solves the “retrofit” problem 
described earlier. Suppose a component has been successfully embedded in many 
contexts, and then an upgrade to it is required. There is a simple way to alter the 
component so that existing embeddings cannot see it, yet all subsequent ones do. 
Each place where alterations are required, instead of replacing what was there, 
place it in a <otherwise> clause and make the new variation a choice in the same 
<select>, parameterized with a variable whose value is time-dependent. Existing 
embeddings do not contain the variable and so will pick up the original frame- 
text, while newer ones will select for the alternative. This technique extends 
to successive alterations by adding further <select> options, and if the frame 
becomes cluttered, simply invoke the <select> from a sub-frame. 

So, no proliferation of look-alike polymorphs occurs, and no retrofits are 
forced for the sake of compatibility. Nor are work arounds needed. The single 
adaptive component exhibits its evolutionary audit trail for anyone to under- 
stand. The dilemma I described has been “de-horned” . 

The last construct I would like to show you iterates a pattern such that each 
iteration expresses a distinct but predefined variation on a theme. For example, 
in designing a GUI there is an archetypal way to take any data field and position 
it within the window, display it, provide help, accept an entry, validate it, provide 
error messages, and so on. The number of data fields and their specific properties 
can be parameterized so that each iteration over the archetype instantiates it 
with a set of properties for each field. The syntax for iterating a pattern looks 
like this 

<while> <;>name</;> <content> frame-text </contentx/while> 

frame-text not only contains the pattern to be iterated but also increments 
a parameter that is part of the name expression such that when the resulting 
evaluation of name is undefined (not found in the symbol table), the <while> 
terminates. 



8 Generators are Unnecessary 

This section’s heading is deliberately provocative. As I said in the introduction, 
generators have had a checkered history. They have been a source of programmer 
frustration because they presume to know how to solve a class of problems 
in advance of knowing all the contexts for which the generator is applicable. 
Programmers have a knack for finding situations where the generated code needs 
to be modified, but either they cannot get at the code or they can but now reusing 
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the generator destroys their changes. I know this first hand, a main reason why 
I was motivated to develop frame technology. 

The fundamental problem is that the generator “hard wires” the domain’s 
semantics inside itself. Adaptive components, on the other hand, provide a means 
of storing semantics externally to the processor that assembles them into source 
code (or whatever). In principle, any detail within any component is adaptable, 
and all adaptations for a given executable can be segregated into one separate 
component (e.g., a specification frame). Instead of building a generator, one 
builds a bi directional translator between the desired high-level abstractions 
(e.g., WYSIWYG tools), and semantically equivalent parameters in a frame. 
From there the assembly process completes the code construction, as would a 
generator, but with the vital difference that the result is arbitrarily fine-tunable 
by a programmer without compromising users of the high-level abstractions. 

In this manner all the power of generators can be made available without 
their drawbacks. 



9 Adaptive Components in Practice 

I have tried to explain how, in theory, adaptive components offer: 

— Meta-models for domains from well- to ill-defined. 

— Automated software modification with increased reliability. 

— The ability to collapse the information needed to span a given domain. 

— Modularity that matches the domain. 

— Flexible software architectures (product lines) . 

— Adaptability across time. 

— Localization of context-sensitive domain knowledge. 

— More forgiving approaches to analysis and design. 

— The means to transcend code generators. 

Good science requires the objective testing of theories. While my firm, Netron 
Inc., has plenty of strong but anecdotal evidence, such claims are routinely dis- 
missed as “vendor hype”. To counter this, Netron took the unusual step of invit- 
ing QSM Associates, a highly respected software metrics firm, to do a study and 
let the chips fall where they may. The study was funded by its participating or- 
ganizations: Ameritech, Chemical Bank (now Chase), Hudson’s Bay Company, 
Noma-Cabletech, Revenue Canada, Teleglobe Insurance Systems, Union Gas, 
and two who wished to remain anonymous. They submitted 15 projects done in, 
believe it or not, COBOL, and ranging in size up to 9.3 million ESLOC (effective 
source-lines of code = new and modified lines without blanks and comment) — 
the average being 841K ESLOC. First published in 1995, the study was updated 
with 14 more projects in 1998. (All figures are courtesy of QSM Associates, Inc.) 

Figure 1 compares the study data against QSM’s database of thousands of 
projects done with all manner of tools, languages, and management techniques. 
The projects are spread along the x-axis, which is QSM’s index of productivity 
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or PL 



= logl.272 



ESLOC 



-26.6 



( 1 ) 



The PI combines the project duration (time from detailed design to user accep- 
tance inclusive) with staff-months consumed (ejfort) and the size (ESLOC) and 
complexity (a skills factor B between 0.16 and 0.39) of the resulting system. It 
is logarithmic such that an increase of 3 Pis implies a doubling of productivity 
— given the same staff and time, the higher-PI project would have turned out 
a system twice as large. The open bars show the distribution of projects from 
QSM’s database; the closed bars distribute the projects in the study. The up- 
dated study shows an average jump of 10 Pis, compared to the industry norm of 
16.9. QSM found that average project time shrank by 70% and staff costs were 
down by 84%. 
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These are statistics that many industry people find hard to believe. Some 
object that ESLOC is a bad measure of size, but all admit that multi-million 
line systems are non-trivial. Others wonder if QSM’s database is outdated. In 
fact, QSM regularly purges old projects. To drive home this point, QSM’s Figure 
2 compares the study data to projects done with 00 tools. Again you can see a 
major displacement between them and frame technology. 

All this nit picking leads me back to a question I raised near the beginning 
— why, in spite of hard evidence of efficacy, are adaptive components not the 
standard way that industry does things? I wonder if frames are an idea whose 
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Fig. 2. Productivity of Frames vs Object Technoiogy 



time is yet to come. When people can shake off their mental bias for building 
things from immutable parts, when they realize that well tested adaptive com- 
ponents, like well tested subroutines, can be trusted to output correct results 
— modified software — much faster and more reliably than humans can, when 
they become responsive to the importance of avoiding brittle software and the 
attendant maintenance headaches, just maybe then they can shift to a rigorous 
development paradigm based on adaptive components. 

If you agree with where I am pointing, then help me make it happen! Thank 
you for listening to my rant. 
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Abstract. Despite advances in software engineering and process methodologies 
over the past decades, not many IT projects seem particularly well adapted to 
today's fast-paced world. Software developers must start to acknowledge 
change and even uncertainty as a given, rather than the exception that should be 
studiously avoided, and they must adapt their techniques accordingly. 

Some business domains have seen attempts to address this situation. Several 
workflow vendors, for instance, have been marketing change and end-user 
programmability as major assets of their products. But, in general, they have 
been strangely ignorant of (good) modem software engineering practices, and 
the results have not really lived up to the claims. But we may expect a revival 
on a grander scale: the ability to (re) define the business logic on the fly is 
becoming a cmcial asset when businesses re-align their core processes around 
the Internet. 

The Internet is transforming the way we envision and design applications. 
While we could build yesterday's simple Web applications with, let's face it, 
primitive techniques, this is simply no longer tme. High-volume databases, 
long-term transactions, interoperability, distributed objects, re-use, these are 
some of the technical issues that must be dealt with. But the real challenge will 
be to leverage all this technology: we must empower the user to set up, 
maintain and change his applications more easily. 

We need dynamic systems, where applications can be changed at mn-time in a 
high-level way, preferably by end-users. Above all, we need appropriate 
architectural techniques. In this paper we explore the use of dynamic object 
models. It turns out that the basic concepts are fairly simple. As for the 
difficulties, we can borrow solutions from many disciplines in computer 
science. If we do it right, we can even make the system work for itself. 



Introduction 

When we analyze a typical business application, say, a relational database application, 
we find out that knowledge of the business model is usually hard to locate. Looking at 
the database structures does not tell us the whole story of object structures and 
relationships. Neither is it easy to spot higher-level constraints and business rules. 
Thus we have to delve into the application code in order to discover the business 
model. This makes it harder to understand the business model, and it makes it even 
harder to change. 
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Fig. 1. Business Logic Hard-Wired in End-User Applications (Typical Database Application). 



In a way, this is all the more surprising since object technology (and later 
component technology) was heralded as the solution to re-use and enhanced 
maintainability. One of the problems is, of course, that object technology itself can 
never make up for lack of design and algorithmic skills, and for inappropriate process 
models and team structures. And we need more than class libraries or re-usable 
components. 

Object-oriented frameworks aim to capture the commonality and variability in 
particular application domains. Frameworks take time and investment to mature, 
however, and are (initially) harder to understand than straightforward code. Hotspots, 
points in the framework where we extend or specialize the generic behavior, are of 
crucial importance. Even with good documentation it usually takes knowledgeable 
developers to properly use or build frameworks, as the hotspots are (once again) often 
buried in code. Another point that is often overlooked relates to the framework’s 
ability to work ‘out of the box’. Many so-called frameworks are really just skeletons 
that do not perform any real function of their own. 



Model-Driven Systems 

In the past decade, several procedure-oriented workflow systems have been replaced 
by dynamic systems, driven by explicit models of the workflow processes. These 
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allow for easy run-time change of process definitions without having to shutdown and 
restart the system. Typical examples are flowchart-like models of processes with 
graphical editors to build and simulate the processes (whether flowcharts are 
appropriate representations of business processes, is beyond the scope of this paper). 

Another example of a model-driven system is the typical CASE tool. But instead of 
being interpreted at run-time, most CASE tools generate source code that is 
subsequently edited by developers. The challenge here is to make design and 
development co-evolve consistently, the biggest obstacle being the usual discrepancy 
between a domain-specific design model and a general-purpose programming 
language. 



Dynamic Object Models 

A system with a dynamic object model (a.k.a. active or adaptive object model) 
[Riehle+2000,Tilman2000] has an explicit object model that it interprets at run-time. 
If you change the object model, the system changes its behavior (essentially) 
immediately. What goes into a dynamic object model is domain-specific. Typically 
the model defines objects and their relationships, their states and the conditions under 
which an object changes its states, as well as the various business rules that specify 
how a company does their business. Commercial and research applications of systems 
driven by dynamic object models have been developed for insurance, banking, 
administrative and workflow applications, amongst others. Several of these add their 
own specific extensions a core dynamic object model theme. 

For dynamic object models to be effective, they need to be sufficiently ‘high-level’ 
and complete with regards to their application domains. The UDP insurance system 
[Johnson-l-1998], for instance, obviates the need for a general programming language, 
since the computations that the users are interested in consist essentially of arithmetic 
expressions, akin to spreadsheet formulas. End-users express their business rules in 
terms of this high-level ‘scripting language’, and in terms of objects and object 
properties. The insurance system empowers end-users to specify the solutions to their 
problems completely in terms of the UDP dynamic object model. 

Generic CASE tools on the other hand provide a high-level way to express objects 
with their classifications and relationships, but necessarily leave out many details that 
must be specified in an external language, usually a general-purpose programming 
language. Thus CASE tool models, even if dynamically interpreted, are not 
necessarily effective dynamic object models. 

Dynamic object models bring ‘programmability’ closer to the end-user, who is the 
domain expert. Still, a bit of caution is in order here, as there are many shades of gray 
in domain expertise and in ‘end-user programmability’. Modeling several hundred 
object types and their relationships, for instance, requires good skills in object- 
oriented design, whereas creating queries by means of a visual builder may be easier 
to learn if the user is not exposed to actual SQL code. 
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Model-Driven Frameworks 

Dynamic object models are high-level, executable specifications of the business 
model. Hence dynamic object models require execution engines. The engine typically 
provides components and tools to manage the dynamic object model in a repository, 
to let users interact with the objects through forms, overview lists and search 
facilities, and to execute the business rules. Thus it makes sense to view the engine as 
a domain-specific virtual machine for programs expressed as dynamic object models. 

When we build several end-user applications with dynamic object models, after a 
while we may discover duplicate pieces of high-level ‘code’. This usually indicates 
the need for a new kind of generic functionality. Then we change the meta-model that 
describes what goes into a dynamic object model, and modify the execution engine to 
support the new functionality. From now on, end-user applications can be developed 
that exploit this functionality directly, without having to ‘code’ it over and over again. 

While dynamic models seem an overly complex solution at first sight, they arise 
rather naturally. Applications tend to evolve over the course of the years. Some 
aspects, like the organizational structure and the business rules, should be easier and 
faster to change than others, such as the generic functionality of form tools. Object- 
oriented frameworks provide generic functionality and support for evolution, but, in 
general, their designs do not pay sufficient attention to the nature of genericity and 
evolution from the user’s point of view. Starting out with a regular framework, we 
make hotspots that should be easily configurable by ‘end-users’ explicit, by means of 
domain-specific dynamic object models. We leave the interpretation of the models to 
the slower-evolving generic components and to the tools of the framework. Such 
model-driven frameworks thus directly support software evolution at different levels. 

Obviously, we can apply this technique recursively. If, say, at the framework level 
we regularly require new interface tools, it may worthwhile to consider if a dynamic 
object model for assembling user interface components is preferable to coding them 
each time from scratch. 



Example: The Argo Framework 

Argo, who are responsible for managing public schools within the Flemish 
community in Belgium, initiated a project to support their administration through IT 
technology. Not only did they commission a set of finished end-user applications, but 
they also required a framework to develop and maintain these applications 
[Tilman-l-1999]. Furthermore, end-users were to participate in the development 
process. All this was partly motivated by the fact that Argo, being a political 
organization, was very susceptible to changes in policies. In the course of the project, 
two major reorganizations occurred. As a result, project goals and requirements 
changed frequently. 

From the onset we opted for an object-oriented framework driven by a dynamic 
object model, although both framework and meta-model as such have changed 
considerably over the past six years. The system (which is completely developed in 
Visual Works\Smalltalk) supports the development of administrative-style 
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applications. These applications typically require a mix of database, document 
management and workflow functionality. In a later phase we extended the client / 
server tools with an application server that gives a new interpretation to the dynamic 
object model and allows users to access the existing applications through the Internet. 
While the actual end-user applications are specific to the school system, the generic 
functionality of the framework itself suits many administrative environments. 

To enable users to participate in the development process, end-user applications 
can be developed to a large degree without coding, through modeling and 
configuring. Scripting is confined to implement more sophisticated business rules. 
Even the scripting elements are partly modeled and stored in a (meta-) repository 
alongside the rest of the dynamic object model, separate from the framework code. 





Fig. 2. Dynamically Consulted Meta-information Stored in Repository (Argo Framework). 

The dynamic object model at Argo not only describes the business model, such as 
the organizational model, the structure of the objects, the authorization rules, the 
constraints and other business rules, but it also describes how the user interacts with 
all this information. This includes amongst others specifications of forms, overview 
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lists, query screens and default behavior of applications. In contrast to, say, tree-form 
painter tools to design user interfaces, we provide users with configuration tools that 
are driven by the model. 

Building an application with the Argo framework does not follow a specific 
sequence of steps. There are some obvious dependencies, however. For instance, it is 
impossible to define constraints on object types that do not yet exist. Other than these 
dependencies, developers are free to perform the actions in any order they want. 



Building an Application with the Argo Framework 

A new application typically starts by extending the existing object model with the 
required object types, attributes, relationships and basic constraints. Then we specify 
which part of the object model we want to use in our new application. We capture this 
information in application environments. Application environments provide a view on 
the database, but they also serve another important function. For each object type, 
attribute and relationship within the environment, we define in a course-grained way 
how we are going to use that particular element: for instance, can we use it in our 
query editor, is it read-only and can we instantiate a given type? We store both object 
model and application environment in the database. 

Then, without any code or code generation, we get a fully functional default 
application, where the generic end-user tools (such as the query editor, the overview 
list and the form tool) adapt themselves automatically to the meta-information in the 
dynamic object model. The query editor, for instance, dynamically retrieves the list of 
searchable object types and properties from the application environment, and presents 
this list to the user. The form tools allow to create and update objects, and to import 
and view documents. Other tools are operational, such as the thesaurus browser and 
inbaskets containing to-do lists. This (and other) functionality becomes immediately 
available to the user, suitably filtered by the specifications of the dynamic object 
model and the application environment. Thus the framework only needs a (partial) 
model to work right out of the box. 

Later on, the user can customize the default applications interactively, for instance 
to build (even) sophisticated queries and store them for later re-use, and to configure 
more appropriate layouts for overview lists and forms. More knowledgeable users 
configure authorization rules, advanced (passive) constraints and active event- 
condition-action rules. The latter perform specific context-dependent actions 
whenever particular events happen. Behavior may be added to the model in order to 
support the scripting used in some of the business rules. 



Advantages 

The advantages of this approach are manifold. For one, we do not need to follow a 
fixed sequence of steps. Instead, we build increasingly complete specifications that 
are available for immediate use. This effectively blurs the boundaries between 
developing and prototyping. Although (due to the generic nature of the framework) 
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we still need a certain amount of scripting, we build end-user applications to a large 
degree by modeling and configuring only. 

Another benefit is the dynamic, interactive nature of the tools, which results in 
immediate feedback. And since the tools adapt themselves automatically to the run- 
time meta-information in the central repository, maintenance of client software 
becomes easier. 



Internet-Enabling the Argo Framework 

The goal for the Internet version was to re-use the existing dynamic object model and 
the generic framework ftmctionality. Since application specifications provide a high- 
level view on how to interact with the information in the database, we had 
considerable freedom to adapt the interpretation of these specifications to the Web 
metaphor. To fine-tune this process even further, we added a new configuration tool 
to the system. But this new tool is really just a configured application, like regular 
end-user applications. This has important ramifications: once the generic end-user 
tools have become available through the Internet, any existing application becomes 
accessible through the Web, including this configuration tool itself This allows us to 
bootstrap the development environment to the Internet. 



Bootstrapping the Argo Framework 

In order for this bootstrapping process to take place, we must not only port the generic 
end-user tools, but also other configuration tools, such as the object model editor, the 
business rules editors and the document management tools. Things get a lot easier 
when the development tools are not dedicated tools, but when they can be configured 
in the system itself, like the new Internet configuration tool. To this end, we model 
the meta-model and system objects like electronic documents, thesaurus keywords, 
layouts, queries and constraints in the system itself Our generic end-user tools also 
need to be sufficiently expressive. It is not a trivial task to design such a system up- 
front, however, so we opted for a bootstrapping process, starting with hard-wired 
development tools, and replacing them gradually as the generic end-user tools got 
more powerful. 

Another benefit is that end-user application developers, who are not experienced 
framework developers can now enhance or personalize their own tools. And 
whenever we enhance the functionality of the generic end-user tools, the development 
tools go along too. 

The ‘downside’ of all this is that we need a more reflective system, including a 
self-descriptive meta-model and rules acting on rules. 
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Fig. 3. Self-Descriptive Dynamic Object Model (Argo Framework). 

Our experience tells us that building such as system is not necessarily that hard, as 
long as we start small and stay focused. In the end, we get many things for free. 



Pros and Cons of Dynamic Object Models 

Dynamic object models allow us to build applications that are easier to change, even 
by end-users. Dynamic object models offer additional benefits, some of which are not 
apparent at first sight, but they also impose their own challenges. We refer the reader 
to the “Dynamic Object Model” pattern paper [Riehle+2000] for additional 
discussions. 
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Advantages of Dynamic Object Models 

Dynamic object models contain run-time meta-information of the end-user 
applications. Since this data is explicitly available, rather than implicitly hard-coded, 
we can put this information to many uses. For one, it becomes possible to 
automatically generate useful and consistent documentation about the end-user 
applications. We can even personalize this documentation for each individual user. 

Compatibility with existing client code is always a difficult problem when 
interfaces need to be changed. Dynamic object models make this much less of a 
chore. Say we must change the tlmctionality of form layouts in the Argo tfamework, 
then we simply write a conversion script that enumerates the layouts in the repository, 
and converts them from one format into another. No ‘traditional’ modification of the 
client code in the end-user applications is required. 

When end-user applications are completely described by (high-level) dynamic 
object models, we can easily migrate to other environments, since we only need to 
port the tfamework code. The end-user applications remain operational. The 
bootstrapping approach used in the Argo tfamework further reduces the amount of 
work that needs to be done. 

Dynamic object models help us set up test suites easily. For instance, by 
enumerating application environments, stored queries and list and form layouts in the 
Argo framework, we easily created a parameterizable tfamework to benchmark 
database servers or to perform regression testing. 

Since the business model is explicitly represented, it is much easier to understand 
the model and to estimate the impact of changing user requirements. Building end- 
user applications becomes a more declarative process, where developers concentrate 
on the ‘what’, rather than on the ‘how’. 



Increased Design and Rnn-Time Complexity 

Frameworks driven by dynamic object models are initially harder to understand. 
Some of this complexity is common to most tfameworks. A further failure to grasp 
readily what is going on is due to the design of dynamic object models. Many systems 
use the following (or a similar) core design: 
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Fig. 4. Dynamic Object Model (Core Design). 

Here, objects are typically represented as dynamically typed instances of a generic 
class. The object types are full-blown objects in their own right. Each object type 
maintains a list of property types that defines the ‘structure’ of its ‘instances’. Each 
instance maintains its properties in a dynamic list. This design enables us to create 
new object types at run-time, and to instantiate these types without having to create 
new classes. More elaborate or domain-specific designs build upon this kernel 
structure, adding, for instance, type-subtype relationships, arbitrary relationships and 
composite structures. 

Hence what used to be an instance of a specific class in a regular programming 
language, is now represented as a collection of objects with various relationships. 
This makes it harder to understand the design and the run-time structure of dynamic 
object models. The main problem lies with the tools. Dynamic object models 
effectively introduce new ‘languages’, but we are still using the old development tools 
(such as class browsers and object inspectors) to look at the structure. Once we have 
created new design and development tools adapted to the new language, these 
problems go away. 

Some of the regular development tools are no longer available when building end- 
user applications. The Argo framework, for instance, uses a multi-user team 
development tool to store the framework code. Since we needed versioning facilities 
for business rules and object behavior, we added a simple versioning scheme to our 
dynamic object model. A bit of modeling and a few business rules enabled us to 
configure this in the system itself in just a few hours. 
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Performance 

Many people fear that dynamic systems are inherently slow. Yet many dynamic 
systems in diverse domains offer the flexibility we really need, while exhibiting good 
performance. Smalltalk virtual machines, for instance, have come a long way since 
the early seventies, and they are still improving; processor speeds have increased in 
part because of clever, dynamic scheduling strategies, and database vendors are 
improving statistics-based query optimizers. 

Whether performance of a specific dynamic object model is acceptable, ultimately 
depends on its particular application domain and on the end-user requirements, but we 
can also borrow many ideas from other software engineering domains. 

For one, there are many ways to improve upon a naive implementation of dynamic 
object models [Tilman2000]. If the language permits, we can even generate concrete 
classes for each object type dynamically, using a just-in-time scheme. 

The Argo framework contains many caching strategies, some explicit, some 
implicit. These are not confined to the persistence component only. 

Another solution aims to optimize for typical usage patterns. The Argo persistence 
component, for instance, uses generic hints and statistical information to drive the 
query generation process. Discovering usage patterns and designing appropriate 
heuristics usually takes time, however, since we need to analyze the run-time 
behavior of a relevant number of end-user applications. 

We can also apply the very dynamic nature of these systems to overcome their 
weaknesses. Some of the authorization rules in the Argo system, for instance, require 
a lot of data, depending on the contents of an object’s properties, and are rather time- 
consuming. By waiting till the last possible moment, we analyze the definitions of 
these rules at run-time, and decide which ones we may safely skip. Partial evaluation 
techniques offer additional interesting opportunities to execute static ‘computations’ 
at development time. 
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Abstract. Over the recent years aspect-oriented programming (AOP) has found 
increasing interest among researchers in software engineering. Aspects are 
abstractions which capture and localise cross-cutting concerns. Although 
persistence has been considered as an aspect of a system, persistence of aspects 
has been largely ignored. This paper identifies several scenarios where aspect 
persistence is an essential requirement. A model for aspect persistence and an 
initial prototype based on Aspect! (0.6beta2) are presented. Various open issues 
are also pointed out. 



1 Introduction 

Over the recent years aspect-oriented programming (AOP) [9] has found increasing 
interest among researchers in software engineering. It aims at easing software 
development by providing further support for modularisation. Aspects are abstractions 
which serve to localise any cross-cutting concerns e.g. code which cannot be 
encapsulated within one class but is tangled over many classes. A few examples of 
aspects are memory management, failure handling, communication, real-time 
constraints, resource sharing, performance optimisation, debugging and 
synchronisation. In AOP classes are designed and coded separately from aspects 
encapsulating the cross-cutting code. The links between classes and aspects are 
expressed by explicit or implicit join points. An aspect weaver is used to merge the 
classes and the aspects with respect to the join points. This can be done statically as a 
phase at compile-time or dynamically at run-time [7, 9]. 

AOP research has identified persistence as an aspect of a system [12, 21]. However, 
the need for aspect persistence has not been considered. Existing work on lifetime of 
aspects [7, 9, 11] has argued that at least some of the aspects should live for the 
lifetime of the program execution and not die at compile-time. In certain cases the 
need for aspects to outlive the program execution can arise. [17], for example, 
proposes the use of an aspect repository for managing and reusing aspects in a 
distributed environment. Aspects in the repository live beyond the execution of the 
programs using them. Some aspects can be persistent by nature. [18, 19] identify 
several aspects cross-cutting the schema and the data in object-oriented databases. 
These include instance adaptation, versioning, clustering, access rights and data 
representation, etc. Due to the close integration between object-oriented programming 
languages and object-oriented databases these aspects are seamlessly used by 
application programs but are persistent by nature and reside in the database. 
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This paper proposes an approach for aspect persistence. The next section makes a 
case for aspect persistence by discussing several scenarios involving persistent 
aspects. A description of the proposed aspect persistence model is presented in section 
3. The model is independent of a particular AOP approach and takes into account the 
evolving nature of aspect languages and representations. An initial prototype 
implementation of the model using Aspect! (0.6beta2) [2] and the Jasmine (1.21) 
object database management system is discussed in section 4 while the various open 
issues are outlined in section 5. Section 6 concludes the paper and identifies directions 
for future work. 



2 Why Persistent Aspects? 

This section describes some scenarios involving persistent aspects. The discussion 
demonstrates that aspect persistence is an essential requirement in many cases, hence 
highlighting the need to provide support for the purpose. 



2.1 An Aspect Repository for Managing and Rensing Aspects 

[17] proposes the use of an aspect repository to manage and reuse aspects in a 
distributed environment. The approach aims at reducing the additional complexity 
introduced due to the need to find an aspect in a network of computing units. Aspects 
in the repository are persistent and applications at different locations can query the 
repository to retrieve the required aspects at run-time. This is shown in Fig. 1 [17]. 




Location C 



Fig. 1. Managing and Reusing Distributed Aspects through an Aspect Repository [17]. 

Another scenario is an automated software development environment where both 
components and aspects reside in a database. The appropriate components and aspects 
are retrieved by the assembling process which carries out the weaving. 
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2.2 Instance Adaptation Dnring Class Versioning 

The second example is based on instance adaptation during class versioning [3, 14, 
20] in object-oriented databases. Class versioning allows several versions of one type 
to be created during evolution. An instance is bound to a specific version of the type 
and when accessed using another type version (or a common type interface) is either 
converted or made to exhibit a compatible interface. This is termed as instance 
adaptation and is essential to ensure structural consistency. A detailed description of 
class versioning is beyond the scope of this paper. Interested readers are referred to 
[3, 14, 20]. The following discussion demonstrates that the instance adaptation code 
cross-cuts the class versions and can be separated using aspects. It should be noted 
that the instance adaptation aspects cross-cut persistent entities (the class versions) 
and are persistent by nature. 
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Fig. 2. Class Versioning in ENCORE. 

We first consider the instance adaptation strategy of ENCORE [20]. A similar 
approach is employed by AVANCE [3]. As shown in figure 2, applications access 
instances of a class through a version set interface which is the union of the properties 
and methods defined by all versions of the class. Error handlers are employed to trap 
incompatibilities between the version set interface and the interface of a particular 
class version. These handlers also ensure that objects associated with the class version 
exhibit the version set interface. As shown in Fig. 2(b) if a new class version modifies 
the version set interface (e.g. if it introduces new properties and methods) handlers for 
the new properties and methods are introduced into all the former versions of the type. 
On the other hand, if creation of a new class version does not modify the version set 
interface (e.g. if the version is introduced because properties and methods have been 
removed), handlers for the removed properties and methods are added to the newly 
created version (cf Fig. 2(c)). 

The introduction of error handlers in former class versions is a significant overhead 
especially when, over the lifetime of the database, a substantial number of class 
versions can exist prior to the creation of a new one. If the behaviour of some 
handlers needs to be changed maintenance has to be performed on all the class 
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versions in which the handlers were introduced. To demonstrate the use of persistent 
aspects we have chosen the scenario in Fig. 2(b). Similar solutions can be employed 
for other cases. As shown in Fig. 3(a) instead of introducing the handlers into the 
former class versions they are encapsulated in an aspect. Fig. 3(b) depicts the case 
when an application attempts to access the age and height attributes in an object 
associated with version 1 of class Person. The aspect containing the handlers is 
woven into the particular class version. The handlers then simulate (to the application) 
the presence of the missing attributes in the associated object. 




Fig. 3. The Aspect-Oriented Instance Adaptation Approach. 



Encapsulating handlers in an aspect offers an advantage in terms of maintenance as 
only one aspect is defined for a set of handlers for a number of older class versions. 
Behaviour of the handlers can be modified within the aspect instead of modifying 
them within each class version. Aspects also help separate the instance adaptation 
strategy from the class versions. For example, let us suppose one wants to employ a 
different instance adaptation approach*, the use of update/backdate methods for 
dynamic instance conversion between class versions (as opposed to simulating a 
conversion) [14]. In this case only the aspects need to be modified without having the 
problem of updating the various class versions. These are automatically updated to 
use the new strategy when the aspect is woven. The aspect-oriented approach has a 
run-time overhead as aspects need to be woven and unwoven (if adaptation strategies 
are expected to change). However, this overhead is smaller than that of updating and 
maintaining a number of class versions. The overhead can be reduced by leaving an 
aspect woven and only reweaving if the aspect has been modified. Details of the 
aspect-oriented instance adaptation approach have been reported in [18]. 



* Such a need can arise due to application/scenario specific adaptation requirements or the 
availability of a more efficient strategy. 
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2.3 Clustering 

[19] identifies clustering as a persistent aspect which cross-cuts the objects residing in 
an object-oriented database. Traditionally it is the task of the database application 
programmer to ensure that related objects are clustered together. However, the task is 
easy only for a small number of objects. As the number of objects that need to be 
clustered increases (it should be noted that the clustering reasons could be different 
for different groups of objects) the programmer’s task becomes more and more 
complicated. The situation worsens if the same object needs to be clustered together 
with more than one group. Considering clustering as an aspect of data residing in a 
database allows managing these complex scenarios transparently of the programmer. 
The programmer can specify clustering as an aspect of a group of objects regardless 
of whether some objects in the group also form part of another group. The clustering 
aspects can then be used by the system to work out an efficient storage strategy. 
Furthermore, if the clustering requirements for an object change the programmer can 
re-configure the clustering aspect to indicate which group of objects should be 
clustered with this object. This helps to manage the physical reorganisation of the 
various clustered objects transparently of the programmer. It should also be noted that 
clustering is not necessarily an aspect of all the objects residing in the database. 
Introducing clustering as an aspect allows only those objects having this aspect to be 
clustered. 



2.4 Other Persistent Aspects 

In addition to the examples presented in section 2. 1-2.3 several other persistent 
aspects can be identified. Versioning is an aspect which cross-cuts both objects and 
classes in an object-oriented database (assuming the system supports both object and 
class versioning). Constraints can be considered as an aspect of the object database. 
Traditionally constraints are specified at the application level or through a DBMS 
service. Considering constraints an aspect of database entities and providing a 
concrete abstraction simplifies their specification and management. Access rights, 
security and data representation can also be regarded as aspects. All these aspects 
cross-cut persistent entities residing in a database and are persistent by nature. 



3 A Model for Aspect Persistence 

Section 2 presented several scenarios where aspect persistence is an essential 
requirement. This section proposes a model for aspect persistence. The model is based 
on the following observations: 

1. Object database management systems often require that classes whose instances 
are to be stored in the database extend a system provided Persistent Root Class. 
Examples of such systems include the Object Data Management Group (ODMG) 
standard [4], 02 [15] and Jasmine [6]. 
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2. Due to proprietary restrictions it is not possible to modify the system classes 
implementing the persistence model of the object database management system 
being used. 

3. Most object database management systems employ the persistence by reachability 
principle. When a transaction commits all objects reachable from a persistent 
object are made persistent. Examples of such systems include the ODMG standard 
[4], 02 [15] Jasmine [6] and Object Store [16]. 

4. Persistence is a cross-cutting concern in a system [12, 21]. 

5. There are various AOP approaches available e.g. AspectJ [2], Composition Filters 
[1], HyperJ [5], Adaptive Programming [10, 13], etc. All these approaches aim at 
achieving a better separation of concerns. However, the aspect structures employed 
for the purpose vary considerably. 

6. Aspect languages and aspect structures are continuously evolving as AOP 
technologies mature. 
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Fig. 4. The Aspect Persistence Model. 

The persistence model is shown in Fig. 4. All application classes extend the Persistent 
Root Class offered by the particular object database management system. A similar 
mechanism is employed for making application aspects persistent. All application 
aspects extend a Persistent Root Aspect. This is a natural extension of the persistence 
model employed by several object database management systems (observation 1). 
Since it is not possible to modify the system classes implementing the persistence 
model of the object database management system (observation 2), all links between 
aspects and classes have been kept strictly class directional i.e. the aspects know 
about the classes but not vice versa [8]. This also facilitates the modification and 
reuse of the aspect code [8] (as discussed later) and avoids introduction of additional 
evolution complexity. 
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When a transaction commits all aspects reachable from a persistent object are made 
persistent (observation 3). As shown in Fig. 5 Aspect 1 is an instance of an aspect 
extending the Persistent Root Aspect and reachable from a persistent object Object 1. 
It is, therefore, made persistent. Although Aspect 2 is also an instance of an aspect 
extending the Persistent Root Aspect it will not be made persistent as it is not 
reachable from any persistent objects in the scope of the transaction. Aspect 3 poses 
an interesting scenario. It is reachable from a persistent object but is not an instance of 
an aspect extending the Persistent Root Aspect. It will be coerced into persistence in 
order to preserve persistence by reachability. The coercion strategy can be to force the 
aspect to extend the Persistent Root Aspect or to annotate the particular aspect 
instance {Aspect 3) with persistence-related code. The choice of coercion strategy has 
been left to the implementation of the model. It should be noted that aspect 
persistence by reachability is transitive in nature i.e. not only aspects reachable from 
persistent objects are made persistent but also aspects reachable from persistent 
aspects are transitively made persistent. 
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Fig. 5. Aspect Persistence by Reachability. 

As shown in Fig. 4 the Persistent Root Aspect encapsulates the persistence by 
reachability code. This code defines the behaviour of an aspect transitively reachable 
from a persistent object upon transaction commit. Since all application aspects extend 
the Persistent Root Aspect the behaviour is propagated down the hierarchy. Reflecting 
on observations 4, 5 and 6 persistence has been regarded as an aspect of the persistent 
aspects (cf Fig. 4). The persistence approach is separated from the persistent aspects 
through the Aspect Persistence aspect. This makes the model independent of a 
particular AOP approach because the persistent aspects do not encapsulate the 
knowledge about their storage structure (which largely depends on the particular AOP 
approach being employed). It also localises the changes resulting from the evolution 
of the aspect language or the aspect structure making maintenance and modifications 
to the persistence model inexpensive. Such changes are further aided by the class- 
directional nature of links between aspects and classes. 
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4 PersAJ: An Aspect! Prototype 

This section presents PersAJ (Persistent Aspect!), an implementation of the 
persistence model using Aspect! (0.6beta2) from Xerox PARC and the Jasmine (1.21) 
object database management system from Computer Associates. The implementation 
is shown in Fig. 6. Application classes and aspects from the instance adaptation 
scenario in section 2.2 have been used as an example. It should be noted that the 
figure only depicts the code relating to the description in section 3. Other 
implementation code has been omitted. 




Fig. 6. PersAJ: An Implementation of the Persistence Model Using Aspect! (0.6beta2). 

PRC is the Persistent Root Class in the Jasmine Persistent Java binding (pj). This 
class is extended by all Java classes whose instances are to be stored in the Jasmine 
database. Since no information is available about the PRC class, we have introduced 
the class PObject which acts as the Persistent Root Class for all application classes in 
PersAJ (in this case Person_Ver_l). PObject has a special instance-level method 
called persistj ) which is invoked for all persistent objects (identified through 
persistence by reachability) just before a transaction commits. This is achieved by 
providing wrappers around the Jasmine transaction commit operation. PAspect is the 
Persistent Root Aspect in PersAJ and is extended by all persistent application aspects 
(in this case Handlers). The application aspects implement a special static advice 
making the aspect instance reachable from the associated class object being 
instantiated. The advice shown in PAspect (cf Fig. 6) determines all reachable aspects 
from a persistent object after the persist( ) method has been invoked (upon transaction 
commit). All reachable aspects are made persistent through a call to the persist( ) 
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method for the aspeet instance. The persist( ) method is introduced into PAspect by 
the AspectPersistence aspect which separates the persistence approach and storage 
structure from the aspects being made persistent. 

The Jasmine Persistent Java binding (pj) offers a Java Persistence Processor (JPP): a 
code generator which adds persistence capability to a Java class by adding code to its 
definition. It also generates a corresponding schema definition for the underlying 
Jasmine database system. PersAJ employs a simple script which runs the AspectJ 
compiler in the preprocess mode. The code generated by AspectJ is passed to a 
custom built parser which parses the generated class definitions and replaces any $ 
signs with two underscores. This is essential as AspectJ uses $ signs to differentiate 
generated code from the Java code supplied by the programmer while Jasmine regards 
$ as a reserved character. The class definitions produced by the parser are passed to 
the Jasmine Persistence Processor which adds the persistence related code and 
generates the database schema. Instances of the processed classes and aspects can 
now be stored in the database. 



5 Open Issues 

This paper introduces the idea of persistent aspects. One of the open research issues is 
the persistent representation of an aspect. Due to the different aspect representations 
e.g. AspectJ, Composition Filters, etc. used in application programs, persistent 
representation of aspects needs careful exploration. 

Persistent aspects and dynamic weaving introduce additional overhead at run-time 
and can be feasible only with efficient weaving mechanisms. The development of 
efficient weavers is therefore an important research issue. 

One of the issues identified during development of PersAJ is the need for 
parameterised aspects. This is not supported in the AspectJ implementation (0.6beta2) 
due to the lack of parametric polymorphism in Java. If aspect parameterisation is 
available the implementation can be more generic and maintainable. An aspect can be 
parameterised by classes in which it is to be woven, hence, making the join points and 
crosscuts generic. Special weave parameters can be used to provide a generic 
reconfiguration mechanism during dynamic weaving. It can also provide a solution 
for the different aspect representations problem identified above. The representation 
to be used can be determined by the aspect passed as a parameter making the 
persistence model more transparent to the programmer. 



6 Conclusions and Future Work 

This paper has proposed an approach for aspect persistence. Aspect persistence is a 
natural extension of existing work on lifetime of aspects. Although existing AOP 
research has considered persistence as an aspect of a system, the need for aspect 
persistence has been largely ignored. The novelty of this work is in identifying 
concrete scenarios where aspect persistence is an essential requirement and providing 
support for the purpose. The proposed aspect persistence model is independent of the 
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aspect language and the aspect representation employed by it. Persistence is 
considered an aspect of the persistent aspects hence providing support for inexpensive 
changes to the persistent representation of aspects due to changes in the aspect 
language or the aspect structure. Class-directional links allow implementation of the 
model without modifying the persistence model of the object database management 
system being used. A prototype implementation of the model based on Aspect! 
(0.6beta2) and Jasmine (1.21) object database management system validates the 
concepts proposed in the paper. 

At present Aspect! 0.7beta4 has been released. Our work in the immediate future will 
involve porting the PersA! implementation to the new release. We are also interested 
in providing implementations of the model for other AOP approaches. At present the 
persistent representation of aspects is handled by the Jasmine Java Persistence 
Processor. We are interested in investigating various persistent representations of 
aspects. We are particularly interested in developing an infrastructure mapping the 
various aspect structures on to a common persistent representation and vice versa. 
Such an infrastructure will serve as middleware between implementations of the 
persistence model for different aspect languages and the common persistent 
representation of aspects. 

Note: The work reported in this paper is part of the Aspect-Oriented Databases 
initiative at Lancaster which aims at bringing the notion of separation of concerns to 
databases and providing aspect persistence mechanisms. Further information can be 
found at: http://www.comp.lancs.ac.uk/computing/aod/ 
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Abstract. Patterns have a longstanding identity in the scientific community as 
results of a phenomenon called symmetry breaking. This article proposes a 
formalism for software patterns through connections from software patterns to 
symmetry and symmetry breaking. Specifically, we show (1) the ties from 
Alexander's work to symmetry and symmetry-breaking foundations; (2) many 
programming languages provide constructs that support symmetry; (3) software 
patterns are the results of symmetry breaking, compensating for design 
shortfalls in programming languages. The proposed pattern fonrialism may be 
useful as a foundation for pattern taxonomies, and to differentiate patterns as a 
design discipline from heuristics, rules, and arbitrary micro-architectures. 



1 Introduction 

Most contemporary attempts to formalize patterns (e.g., [14], [15]) ignore both the 
prior art and the most relevant foundations for potential formalization. This paper 
shows that symmetry lies at the very foundation of the pattern formalism. The paper 
also shows that the symmetry formalism has historically been broad enough not only 
for the natural sciences and Alexander's work, but that it can serve software as well. 

Pattern and symmetry are closely related. The formation of pattern can be 
characterized by symmetry operations, which, in the sense of classic symmetry, are 
rigid motions of geometric object [33]. We believe that the geometric nature of classic 
patterns has been the main inspiration of Alexander’s work ([1], [2], [3]). An early 
realization of our research is that most of these structural properties relate to 
symmetry. Leading software pattern practitioners have also recognized that the best 
software patterns are also geometric in appearance and organization for reasons of 
composability, which is in concert with Alexander's theories of patterns and centers. 

The next section introduces symmetry concepts and group theory, which is the 
prerequisite for the paper. Section 3 gives an overview to Alexander’s theory and its 
connection to symmetry. Section 4 presents symmetry in 0-0 software. Sections 5 
and 6 introduce symmetry breaking as a foundation for patterns. Section 7 illustrates 
symmetry and symmetry breaking in programs and designs. Section 8 proposes a 
formalism for software patterns. We conclude the paper in Section 9. 
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2 Group Theory and Symmetry Concepts 

Group theory offers constructs to formally characterize symmetry through symmetry 
groups. This section gives a brief introduction to group theory and symmetry ([28], pp 
9-10): 

A group is a nonempty set G together with a law of composition 

(a, b) ! ab : G X G ^ G satisfying the following four axioms: 

1. Closure. For all a, b € G, the set G is closed under composition: ab, ba e G. 

2. Associativity. For all a, b, c e G, the composition is associative: (ab)c = a(bc). 

3. Existence of An Identity Element. For all a e G, there exists an element e € G 
such that ae ^ a ^ ea . 

4. Existence of Inverses. For each a g G, there exists an a* g G such that aa* = e 
= a"'a. 

A symmetry of an object is a transformation that leaves the object apparently 
unchanged ([30], p.28). Classic symmetry transformations are rigid motions, such as 
reflection, rotation, translation, and their combinations. For example, the appearance 
of the human body is invariant under reflection such that the distances from any 
origin point to its mirror image is preserved with respect to the reflection center. 

Symmetry is more fundamentally about invariant change, i.e., change, yet the 
same. In geometric sense, it means that objects don't stretch or deform as they are 
translated or rotated. This basic idea extends the possibilities of symmetry beyond 
geometric objects. For example, symmetry principles in physics, such as gauge 
symmetries, obey symmetry properties but are not strictly geometric. 

Rosen ([28], p.2) defines: "Symmetry is immunity to a possible change" in the 
context of the broad field of natural sciences. Immunity to the change means that 
some transformation brings the object into coincidence with itself Symmetry means 
invariant change or transformation invariant. Rosen defines a symmetry 
transformation as a bijective (one-to-one and onto) mapping of a state to an image 
state equivalent to the object state ([28], p.80). State equivalence is defined by an 
equivalence relation for a state space of a system, such that any two states satisfy the 
conditions of reflexivity, symmetry, and transitivity. A symmetry transformation S 
can be denoted as: 

u — - — >v = u or S(u) = V = u 

for all states u ([28], p.80). 

In the above definition, v = u denotes the equivalence relation between the states v 
and u. A subspace comprises a subset of states of a state space. An equivalence 
subspace is a subspace within which all the states are equivalent to each other. This 
leads to a general definition of symmetry group: 

The set of all invertible symmetry transformations of a state space of a system 

for an equivalence relation forms a group, a subgroup of the transformation 
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group, called the symmetry group of the system for the equivalence relation 

([28], p. 80). 

A symmetry group doesn't comprise objects (buildings, software objects) of a 
system, but rather a set of symmetry transformations on these objects. The 
symmetries we investigate in this paper are those that are expressed by programming 
language constructs. The symmetry groups will be the compositions of symmetry 
transformations embodied in programming language features with respect to a certain 
set of invariants. 



3 Alexander’s Theories 

Work on software patterns was largely inspired by Christopher Alexander's pattern 
research and application in the fields of architecture and urban design. We 
summarize Alexander’s work here. 



3.1 A Geometric Basis 

As an architect, Alexander deals with geometry. Alexander views geometry to be the 
essence of what he was doing and this view is in line with Klein’s: 

... in Klein’s words, ‘geometrical properties are characterised by their invariance under a 
group of transformations.’ Each type of geometry has its own group ... Group theory 
provides the common ground that links different geometries ([30], p. 44). 



3.2 Pattern 

Alexander's publications on architecture in the late 1970s focused on design elements 
called patterns [3]. Those patterns formed a pattern language, which can be 
partitioned into numerous smaller pattern languages [1]. The goal of pattern 
languages was to contribute to “the quality without a name,” a deep feeling of 
architectural excellence suitable to a given culture 

A pattern is a description of an architectural relationship between design parts. It 
is also an architectural transformation that integrates parts into a larger whole. 
Alexander’s patterns are geometric in nature. A pattern language allows a builder to 
build a house by applying one pattern at a time, in sequence. 



3.3 Theory of Geometric Centers 

While the people using pattern languages had produced good community houses, 
Alexander soon realized that pattern languages alone were inadequate to achieve the 
beauty he sought. There were two problems: the economic processes of building 
didn't support piecemeal growth, and the people using the pattern language had the 
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skills necessary only for gross scales of beauty, not the very fine artisanship necessary 
for wholeness at all scales. These concerns prompted him to develop a new theory 
consistent with, but fundamentally different than, the theory of patterns. This theory 
is based on geometric centers and a piecemeal growth process. Informally, a center is 
something that draws the eye, a geometric region that we notice. Centers combine to 
form geometrically attractive configurations. Patterns usually are stereotypical centers 
that carry an aesthetic — likely culturally attuned — in addition to any visceral beauty 
they may have. Centers drive more at the visceral beauty of pure geometry. 

The process for building with centers is a simple process of structure-preserving 
transformations. One finds the weakest center in a whole and strengthens it by 
adding new centers that make it more whole. If the overall result is more whole, then 
the process iterates to the next center. Each of these transformations increases 
wholeness while preserving the structure of the whole, though there are adjustments 
in the details. A given transformation can clean out a center that has become too 
messy, but the overall structure is preserved. 

In mathematical terms, a structure -preserving transformation is called a 
homomorphism ([28], p.23). Briefly, a homomorphism is a many-to-one mapping 
from one group to another. If a homomorphism is bijective, i.e., one-to-one and onto, 
it is called an isomorphism. So an isomorphism is a bijective homomorphism. 
Although homomorphism preserves structure, homomorphic groups do not have the 
same structure unless they are isomorphic. 

Each transformation strives to strengthen a center by reinforcing one or non- 
orthogonal structural properties in the whole, including 15 structural properties like 
Levels of Scale, Alternating Repetition, Local Symmetries, and Deep Interlock and 
Ambiguity [2]. We note that most of the structural properties are directly linked to 
symmetry. For example. Deep Interlock and Ambiguity and Alternating Repetition 
exhibit bilateral symmetry, and Echoes exhibit translational symmetry. Many of these 
structural properties show through in Alexander's patterns as City-Country Fingers [1] 
which is an example of Deep Interlock and Ambiguity . Alexander emphasizes this: 

There is a profound connection between the idea of a center, and the idea of 
symmetry. 



1. Most centers are symmetrical. This means that they have at least one 
bilateral symmetry. 

2. Even when centers are asymmetrical, they are always composed of smaller 
elements or centers which are symmetrical ([4], p. 42-43). 



4 Symmetry in Object-Oriented Software 

In this section, we explore some examples of symmetry groups in object-oriented 
programming languages. 
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4.1 A Geometry Basis for Software 

Gabriel posited in 1996 that geometry, for us in computer science, translates to the 
structure of the code ([7], p. 34). That is one geometry of a program, but there are 
others, including its modular [8] and temporal [9] structure. Coplien's more recent 
work [13] attempted to establish a geometric basis for C++ idioms for types whose 
operations have inverses. That effort was an attempt to bring some of the popular 
Design Patterns [16], which draw in part on those idioms, better in line with 
Alexander's geometric theories. 

Here we attempt to explore symmetry in software from the perspectives of group 
theory. And interestingly, if one goes back before patterns to the very basics of 
polymorphism, one finds applicability of group theoretic foundations for object 
orientation to be strikingly strong despite the fact that no popular link ever joined the 
two fields. Consider this quote from a plant physiologist in a math journal circa 1986: 

...new theories of symmetry treat as equal also such objects (such equalities) 
which were considered as essentially different in previous theories 
(respectively, as inequalities). The unique reason why these equalities have 
been adopted is always the same thing, i.e. the existence of real or/and mental 
operations making the objects O, compared in features F, indistinguishable. 
([32], p. 396) 

This is almost a textbook definition of (object-oriented) polymorphism. 



4.2 Classification as Symmetry 

A class in an object-oriented program defines a symmetry. The common knowledge 
of the class concept is related to abstract data type and encapsulation. However, we 
can also say that a class classifies objects. A class establishes an invariance 
relationship between the class and its objects and makes all its objects analogues with 
respect to the class structure. Class data and functions remain valid to all the objects. 
A class in this sense defines an analogy, which is a symmetry ([28], p. 164). 

We can validate the symmetry of a class using Rosen’s definition. Recall from 
Section 2 that symmetry is immunity to a possible change. We can analyze the two 
aspects of change and invariance as follows: 

1. Change. A class can be applied to more than one object; the objects to which 
it is applied can be switched from one to another. 

2. Invariance. The class structure is immune to the switching-abouts of objects, 
i.e., what it is true remains true. 

Recall also from Section 2 that changes are represented by transformations. The 
above shows that the order of the objects in a class is immaterial. Mathematically, we 
can represent the changing orders, switching-abouts as a set of transformations. We 
can show that these transformations form a group. For example, changing object O/ to 
02 to 03 is equivalent to changing from O; to 03 . This is the closure property of the 
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group. Changing is associative, in either direction (inverses); no changing is the 
identity transformation. A formal proof of these four properties will appear in a 
separate paper. 

A linguistic realization of such transformations is the copy constructor, which 
satisfies the four properties of the group. More importantly, such transformations are 
implicit, and the fact that a user can create more than one object without worrying 
about the sequence of the creation owes to the symmetry of the class. 

It should be noted that many language features — inheritance, subtyping, 
overloading, and others — are ways of expressing classification. By the above 
analysis, all such features are related to symmetries. 



4.3 Inheritance as Symmetry 

Cook and Palsberg [5] define inheritance as an operation on generators that transform 
a base class into a (distinct) derived class. In this section, we only consider one use of 
the inheritance operation, i.e., the use of inheritance for type derivation or subtyping. 
Only then can we say that inheritance preserves the structure of the base class. Other 
uses of inheritance, such as for implementation convenience or function overriding, 
do not preserve the class structure and therefore will not be considered here. 

When inheritance is used for subtyping, it creates a type hierarchy [22]. A type 
hierarchy consists of subtypes and supertypes, where objects of a subtype provide all 
the behavior of objects of its supertype plus something extra [22]. A type hierarchy 
satisfies the Liskov Substitution Principle (LSP): 

If for each object Oi of type S there is an object 02 of type T such that for all 
programs P defined in terms of T, the behavior of P is unchanged when Oi is 
substituted for 02 , them S is a subtype of T. [22] 

That is, there are invariants (in this case, behavioral invariants) that hold for a 
program under a transformation that substitutes objects of type S for those of type T. 
Behavior is symmetric with respect to subtyping — only here, the constancy of 
behavior is itself the litmus test by which subtyping is judged, not vice versa. 

Given a subtyping path in a type hierarchy, we say that classes belonging to this 
path are equivalent in that they have the same invariants as the base class. We define 
all the classes in such a single path as a set or a state space as per Rosen’s definition 
in Section 2. We define an equivalence relation for this set as the base invariant 
equivalence, denoted as =, such that it holds for any pair of the classes, e.g., x, y, z, 
in the set: 

1 . Reflexivity. x = v for all x (Every class of the set is equivalent to itself). 

2. Symmetry, x =v for all v, y (Any pair of classes is equivalent). 

3. Transitivity. X =y,y =z =» x = z for all v, y, z. 

In Section 2 we give the definition that a symmetry transformation is a bijective 
(one-to-one and onto) mapping of a state to an image state equivalent to the object 
state in a state space for an equivalence relation. Here the state space is a set of 
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classes of a given subtyping path. States in this state space correspond to classes. We 
represent an inheritance operation or derivation as D such that: 

X — — — = X or D(x) = y = X 

for all classes x in the set. 

We now prove an inheritance operation is a symmetry transformation. Consequently, 
we need to prove (1) an inheritance operation is invertible and (2) the set of 
inheritance operations for a set of classes as defined above form a symmetry group. 

To prove the invertibility of the inheritance operation, we need to show: 

/) “ 1 —1 
y > X = y or D (y) = X = y 

for all classes y in the set. 

What is the inverse of the inheritance operation? Programming languages are 
graced with pragmatics that are not pure theory, and there is rarely a corresponding 
language realization of the operation that derives from a subtype to a supertype, i.e., a 
supertype cannot inherit from a subtype. However, for example, the (now vestigial) 
class slicing feature of C++ is a way to restore the original state of the system when 
an instance of some class is supplied in a context where a less derived class is 
expected. From this point of view, a slicing operation corresponds to or plays a role of 
inverse. In other programming languages, inverses might be implemented through 
conversion operators or constructors (e.g. in Smalltalk, many algebraic types respond 
to messages such as as Integer). 

Since our goal is to establish a formalism for patterns, we can still define an 
inverse of an inheritance operation as a mathematical operation whose linguistic 
support is missing for practical reasons. Hence we have the inheritance operation 
whose inverse remains to be mathematical. 

We shall now prove that the set of all invertible inheritance operations on the 
classes in a given subtyping path forms a symmetry group. 

1 . Inverses exist in the sense of the above assumption. 

2. Closure follows from transitivity of the equivalence relation. For the inheritance 
operation D on all classes x, y, and z, the composition of DD is the result of 
consecutive application of D, such that: 



X =y = D(x), 

y^z- D(y) = D(D(x)) - (DD)(x), 
X = z = (DD)(x), 

X = z = D^(x) 



Thus for the inheritance operation D their compositions D^, ... are also 
inheritance operations. 
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3. Associativety holds for composition by consecutive application. It is evident that 
using the closure property, we can obtain: 

D(DD) = (DD)D = 

Hence we have proved the associativity. 

4. The identity transformation is a null inheritance operation. In other word, it is a do- 
nothing operation, such that: 

X — - — = X or I{x) = X = X 
for all classes x in the set. 

All the invertible inheritance operations on the classes through a single subtyping 
path form a symmetry group for class invariants. We may consider inheritance 
operations as translations that transform classes in time [33]. We can represent such 
translational symmetries as a linear train of iterations: 

... = D'’d ‘, D~’, D" D, = DD, = DDD, ... 

Translation and other temporal symmetries (transformation over time) are common 
themes in computer programs, as we shall also see in Section 4.4. In fact, temporal 
symmetry is a fundamental phenomenon in symmetry breaking in the natural 
sciences. For example, the idea that time is reversible is called the T symmetry in the 
classic CPT symmetry model of physics; symmetry breaking in all of these 
symmetries is the very reason that anything exists ([19], pp. 79-81). 

Generalizing the ideas of classification and temporal symmetry, we can define 
symmetry in terms of any invariant or notion of equality ([32], pp. 395-396) for a 
system under consideration. 



4.4 Overloaded Operators as Symmetries 

Operator overloading, and associated friendship relationships in C-l-l-, have 
consciously been provided to express symmetry ([21], p749). Overloaded arithmetic 
operators support reflection symmetry, freeing the user from distinguishing between 
the left and the right operands, as A-l-B = B-l-A and A*B = B*A. 

Overloaded operators not only can support reflection, but also translation as well. 
For example, overloaded “/” free the user from distinguishing between 

primitive numbers and objects. 

5 Symmetry Breaking and Pattern 

When a system encounters stress it loses symmetry. The phenomenon of symmetry 
breaking may be explained in terms of symmetry groups. Consider the common 
example of a proto-planet rotating in space, a sphere of symmetry group 0(3). If it 
spins too fast it may become pear-shaped, losing one degree of symmetry so as to fall 
into 0(2). If the spinning continues it may break into a planet/moon system which 
still has overall symmetry 0(2), though it has lost the spherical symmetry of the 
original system. This is called spontaneous symmetry breaking in physics [24], or 
symmetry -breaking as we call it here. Symmetry doesn't really "break", but is just 
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reduced or redistributed in the effect produced by symmetry in the cause. When a 
symmetry group is broken, it results in a new group that is a subgroup of the original. 
Note in this moon-formation example, each of the parts still has 0(3) symmetry, even 
though the system has been reduced to 0(2). 

Symmetry breaking results in patterns, or more precisely, reveals patterns, for too 
much symmetry isn’t perceived as pattern [30]. For example, a regular n-sided 
polygon belongs to the dihedral group D„. When n is infinitely large, the polygon is 
close to a circle, which isn't rich in structure. But if some of the symmetries in this 
polygon break, or when n becomes smaller, say, 6, 5, 4, or 3, we begin to see the 
shape of the polygon. The phenomenon of symmetry-breaking was first discovered in 
1923 by Ingram Taylor when studied the dynamics of the fluid flow, which was 
known as hydrodynamic “symmetry paradox”: 

This paradox, that symmetry can get lost between cause and effect, is called 
symmetry-breaking. In recent years scientists and mathematicians have begun 
to realize that it plays a major role in the formation of patterns. 



From the smallest scales to the largest, many of nature's patterns are a result of 
broken symmetry; and our aim is to open your eyes to their mathematical unity. 
([30], p. xviii) 

The idea of symmetry breaking carries through to Alexander's patterns. Varied 
Ceiling Heights [1] is a symmetry -breaking pattern. Light on Two Sides of Every 
Room [1] is another example: a perfectly symmetric room would have either no 
windows or would have windows on four sides. A room with windows on four sides 
would be perfectly symmetric and would lack the quality Alexander seeks in his 
work. He remarked that too much symmetry is a bad thing: 

Living things, though often symmetrical, rarely have perfect symmetry. Indeed 
perfect symmetry is often a mark of death in things, rather than life. 

I believe the lack of clarity in the subject has arisen because of a failure to 
distinguish overall symmetry from local symmetries. ([2], The Phenomenon of 
Life, 44.) 

and: 

In general, a large symmetry of the simplified neoclassicist type rarely 
contributes to the life of a thing, because in any complex whole in the world, 
there are nearly always complex, asymmetrical forces at work — matters of 
location, and context, and function — ^which require that symmetry be broken. 
([2], The Phenomenon of Life, 45.) 

Symmetry breaking also plays an important role in the characterization of 
Alexander’s 15 structural properties. Local Symmetry requires some symmetry be 
broken. Roughness, Gradients, Echoes, and Levels of Scale all exhibit symmetry, but 
lack perfect symmetry characteristic of the next largest symmetry groups (perfect 
smoothness, bilateral symmetry, and equal size). 
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The idea of symmetry breaking also extends to software patterns, as we shall see in 
Sections 6 and 7. For example, class extension is a symmetry; Bridge is a pattern that 
reflects symmetry breaking under the stress of certain design desiderata. 



6 Patterns and Symmetry Breaking in Software 

As discussed in Section 4, many programming language constructs express symmetry. 
When those constructs fail to solve design problems, programmers often resort to 
patterns to express the design in the programming language. This section attempts to 
show that most software patterns are a result of symmetry breaking. 



6.1 Breaking Type Symmetry 

Assume we have a class, List<T> in some library. We wish to derive from it a new 
type, Set<T>. Let’s see if such derivation can preserve class invariants in List<T>, 
as discussed in Section 4.3. 

First, a set is not a subtype of a list, nor is the reverse true. For example, by the 
principle of extensionability, sets with the same members are equal [17]. So adding 
the same member twice to a set is the same as adding the member only once. 
However, adding the same member twice to a list is different from a single insertion. 

Therefore, deriving a set from a list breaks the symmetry that subtyping attempts to 
conserve. We start with a List and want to transform it into a Set. The symmetry 
breaks. Where does it go? 




Fig. 1. Inheritance Symmetry. 

Imagine if the symmetry did not break, i.e., in the normal subtyping situation, the 
relationship between a subtype and its supertype would be denoted as in Figure 1. 
However, since the symmetry is broken, the new relationship lacks the symmetry of 
Figure 1, reflecting only local symmetry (Figure 2). This is known as the Adapter 
pattern. In this pattern, the insert and has methods appear in all three classes. 
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Fig. 2. Adapter Pattern. 



Most of the struetural patterns {Adapter, Bridge, Composite, Decorator, and Proxy) 
[16] deal with the tension between language constmets that express symmetries, and 
the small perturbations that make those constructs unsuitable for use. Some of the 
behavioral patterns {Iterator, Memento, Observer, State, Strategy, Template Method, 
and Visitor) can also be easily described in terms of temporal symmetry breaking. 
Other design patterns take more imagination to describe in symmetry-breaking terms, 
which raises the question of whether they are patterns in the Alexandrian vein. 

From the perspective of symmetry breaking, a pattern is a way of redistributing the 
forces in a symmetric system in light of some instability. Symmetry is reduced, but 
not lost; it is redistributed in the pattern. The exact way in which it is redistributed is 
difficult to predict. Global symmetries are lost, but local symmetries are preserved. 

We want to point out here that the term local symmetry is not an original 
Alexandrian formulation, but is a standard term in the symmetry group communities 
of crystallography ([23], p. 29; [29], p. 567) and quantum physics ([24], p. 173). 



6.2 Breaking Function Symmetry 

Multiple dispatch is a form of symmetry breaking. In a Liskov type hierarchy, subtype 
methods have a level of equivalent semantics defined by the bounds of pre- and 
postconditions, semantics that programmers usually associate (however informally) 
with the method selector. In a classfiil language these symmetries in a type hierarchy 
preserve class invariants, as discussed in Section 4.3. 

When a type hierarchy is used for dynamic binding, a member function is chosen 
according to the type of its instance at run time. The symmetry breaks if the member 
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function depends on the types of more than one object; the LSP no longer holds. . 
This is a form of symmetry breaking for which the longstanding linguistic solution is 
multiple dispatch. This of course is the Visitor pattern [16]. 

The "Promote and Add" pattern is also symmetry breaking. The “+” operator is 
interesting not only because it supports symmetries as discussed in Section 4.4, but 
also because it has a highly symmetric signature: 

TxT^T 

If we picture “+” operating between pairs of objects, where each object participates 
in a class hierarchy, we get a nicely symmetric picture as in Figure 3. 




Figure 3. Bilateral Symmetry 

But reality is messier because of the need for heterogeneous addition. For 
example, it makes sense to ask for 1 + 4.5, as in Figure 4 — a broken symmetry (and 
it's independent of multiple dispatch or polymorphism). 

The solution is a geometric transformation, a pattern, called Promote and Add, 
originally written up as a C++ idiom [13], but which can also appear in broad 
architectural contexts as patterns such as Add a Switch [9]. It adds local symmetries 
by promoting heir types to their parent types as a prelude to addition, thereby 
reducing the problem to that of the first picture. It is a structure-preserving 
transformation that adds centers (the transformations from derived types to base 
types) to the first picture. The resulting Smalltalk bears out the pattern explicitly in 
methods such as sumFromInteger, asDouble, and 
ArithmeticValue>>retry. 
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Figure 4. Symmetry Breaking in + 



6.3 Breaking Class Symmetry 

We usually think of types as abstract specifications with classes as their language 
realizations. The class structure often follows the type structure. But sometimes, 
even though the type structure reflects a structure -preserving transformation 
consistent with the LSP, the class structure does not preserve structure for reasons that 
owe to language implementation peculiarities. This means that the symmetry of the 
class structure is broken, and a pattern usually results. 

Again consider class Complex, whose implementation is two real numbers. 
Abstractly we can talk of the symmetry group of type Complex as comprising 
transformations that preserve class invariants as discussed in Section 4.2. These 
invariants hold, with respect to member tlmction behavior (type symmetry) for heir 
classes such as Real and Integer, as one would expect. 

Even though the member functions are preserved in the transformations from 
Complex to derived types, the structure of the Complex class may not be preserved 
in Real and Integer. In particular. Complex has two reals and Real would 
have only one. Eiffel would call this restriction inheritance ([27], p. 826). In the 
representation of the class, there is at best a weakened symmetry between Complex 
and Real (one could argue that they are symmetric with respect to the preservation 
of the real component) but all structure is lost by the time we get to Integer. 
Symmetry slowly breaks as we descend the hierarchy. We find the same case for 
Ellipse and Circle, as in Meyer ([27], p. 467). 

In either of the cases, we lose symmetry when the inheritance transformation is not 
structure preserving as required in the LSP, so we have symmetry breaking. What 
results is usually a pattern; in this case, something like Bridge might be in order. 
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7 Other Examples 

We can look beyond object-oriented programming language constructs to find many 
examples of symmetries and symmetry breaking in software design. 

Symmetries abound in programs and can be found beneath almost all programming 
structures. In addition to symmetries we have discussed in Section 4, loops are spiral 
time symmetries. Simple conditionals can be viewed as symmetry breaking, while 
case statements can also be viewed as symmetries that hold the entry and exit points 
invariant. Argument defaulting is a form of overloading where the symmetry is the 
preservation of some argument types. Rather than focusing on these programming-in- 
the-small symmetries, in this paper we focus on the design structures that are of 
greater interest to the field of object orientation. 

Patterns are introduced as a result of symmetry breaking in all of these cases. We 
showed above, in Section 6.2.2, what happened when the symmetry of overloaded 
operator -I- broke. This is hardly an object-oriented phenomenon, either. The work in 

[10] introduced the notion that functional and applicative languages may succumb to 
these analyses more readily than object-oriented languages do, owing to the largely 
geometric nature of the source and translated program structures. 

The symmetries supported by language constructs might be used as a basis for 
objectively comparing the relative expressive power of a programming language. For 
example, the number of symmetries and the type of symmetry might be used as such 
an objective indicator. We suggest further empirical work in this area. 

We find patterns outside the programming language context, too. Patterns such as 
Half Object plus Protocol (HOPP) [25] are an obvious example of symmetry- 
breaking reminiscent of bifurcation symmetry. It should be noted that there is nothing 
fundamental about the system in which HOPP is embedded that suggests that 
symmetry break in a way that results in two objects; it could just as well break into 
three objects, as it does in the pattern Three-Part Call Processing [9]. 

In the Cascade pattern for a public transport system [34], it was observed that both 
the driver duty object and its builder were cascades. A driver duty can be seen as a 
result of a translation of its builder (the Prototype pattern) or as a rotated bilateral 
symmetry. Other examples in telecommunication software are documented in [9]. 

ModeTView-Controller expresses symmetry between a Model object and a User 
object. What breaks the symmetry is the need for the User object to maintain 
multiple views of the Model. That gives rise to the View object itself, and to the 
Controller as a dispatcher from the User to the Model and View. [Trygve Reenskaug, 
Personal conversation, June 1999]. 

Fresh Work before Stale [26] is a classic example of temporal symmetry breaking. 
The pattern creates a broken symmetry in the flow of work that gives priority to new 
work over pending work, and throughput increases as a result. 

Factory Method and Abstract Factory [16] break the symmetry of the structures of 
the object produced by instantiation processes. Template Method breaks the 
symmetry of individual algorithms. 

Domain analysis techniques are rooted in axioms of commonality and variation 

[11] . Group theory may provide constructs for formalizing this structure as well: 
symmetry is about commonality invariance; symmetry breaking is about variations. 
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8 Towards a Pattern Formalism 

So, what is a pattern? The question is the closest thing to a religious debate in 
computer science since the debate raged about “What is object orientation?” There 
are many qualities often used to distinguish, or sometimes distance, patterns from 
mainstream academic computer science. Among these are beauty, maturity of ideas, 
and attunement to human comfort. Here we try to put these elements in perspective. 

Does Alexander's stipulation that a pattern only exists within a pattern language 
([3], p. 312) relate to symmetry and symmetry breaking? Remember that a pattern is 
a transformation, a transformation that breaks some symmetry present in the original 
system, adding asymmetry. The original system defines context; without that context, 
there may in fact be no symmetry to be broken. What might otherwise pass as a 
pattern, out of context, may simply be a symmetry. We can also explain the 
relationship between symmetry and symmetry breaking as a causal relationship: the 
original system exhibiting the original symmetry is the cause; the transformed system 
in which some symmetry is broken is an effect. 

As mentioned early, broken symmetry doesn’t necessarily mean asymmetry unless 
all the symmetry is lost. Although it is not the topic of this article, we wish to point 
out that symmetry only exists with respect to asymmetry ([28], p. 161) and 
asymmetry is a frame of reference to symmetry. 

One may criticize the design patterns [16] as not meeting the Alexandrian criterion 
of necessarily emanating from a whole, from a pattern language. However, some of 
the design patterns constitute patterns that form a pattern language for a particular 
design problem. Symmetries that these patterns attempt to preserve or break may be 
identified. This leaves hope that the design patterns may be legitimized as patterns in 
a fashion tantamount to, or at least analogous to, the closely related idioms work [13]. 

One key suggestion for future work is to show that a pattern language forms a 
contextual framework for the formalization of symmetry breaking (the very definition 
of individual patterns), and that such a framework forms not only a language, but also 
an algebra of specification. The outcome seems likely; after all, a geometry is 
essentially an algebra of symmetries. 

Based on the above analysis, we posit the following formal definition of a pattern: 

Definition: A pattern characterizes a structure resulted from breaking an 
original symmetry, where the symmetry is defined in terms of an invariant in a 
system. Examples of such invariants include class structure (in the vulgar sense 
as in Section 4.2) and behavior (subtyping as discussed in Section 4.3). 

In other words, a pattern represents a symmetry effect that is less symmetric 
than the symmetry cause, where the symmetry cause is produced by a 
programming language construct. Patterns precipitate from symmetries in 
response to both internal and external forces that are the analogy of instability. 

A pattern is a design solution that language constructs fail to provide. A 
pattern language defines a composition law for composing patterns in a specific 
way to solve a larger design problem. A pattern is in relation to a symmetry that 
in turn is in relation to a geometric context: a symmetry group. 

The formal aspect of the definition is necessary but not sufficient. The 
articulation of a pattern is also subject to a value system, and in particular to 
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aesthetic and quality considerations. Among these considerations are human 
comfort, utility, beauty, durability, and maintainability. 

This definition is simply a distillation of the findings of this paper, cast in a way 
that ties the group theoretic foundations with the vernacular definitions. We propose 
it as a foundation for further work in the formalization of software patterns. 

One might use this definition to conclude that because the design patterns reflect 
different geometric contexts, they could never all be unified in a single pattern 
language. For example, the context for Observer and Memento is temporal symmetry 
while the context for most other patterns is more spatial. This contextual modeling 
might provide a framework for a pattern taxonomy that would be more useful than the 
current taxonomy (creational, structural, etc.) for the stated purpose of a pattern 
language: to create whole systems through a piecemeal growth process. 



9 Conclusion and Acknowledgements 

Other attempts to formalize patterns have striven for implementation automation. 
That is not our goal here, as automation of pattern assembly is an oxymoron from first 
principles. Our goal is to provide a formal basis for the characterization of patterns in 
specific contexts. For example, the symmetry theory foundation for patterns explains 
why multiple dispatch is a pattern in Smalltalk, and is not a pattern in CLOS. It 
explains why inheritance for subtyping is a symmetry, lacking the symmetry-breaking 
properties that are characteristic of patterns, even though Inheritance wrongly appears 
as a pattern in some collections [31]. This model may provide a foundation for 
legitimizing other patterns on the basis of temporal symmetry breaking models, even 
hopelessly non-spatial patterns. 

This paper deals with patterns commonly associated with design. There may be 
other useful patterns of programming, such as indentation style and other modular 
constructs, that also exhibit symmetry and symmetry breaking and which are equally 
important to the success of software development. 

Ralph Johnson provided initial feedback to the original draft and has continuously 
provided valuable insights and comments to various versions of the paper. We are 
also deeply indebted to Michael Benedikt, Ellie D'Hondt, Maja D'Hondt, Ed Remmel, 
Curtis Tuckey, Gottfried Chen, Edith Adan-Bante, Peter Mataga, Rachel Suddeth, 
Kevlin Henney, Neil Harrison, Rich Delzenero, Christoph Koegl, and Peter Grogono. 
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Abstract. The composition of software units has been one of the main 
research topics in computer science. This paper addresses the composi- 
tion validation problem evolving in this context. It focuses on the compo- 
sition for a certain kind of units called aspects. Aspects are a new concept 
which is introduced by aspect-oriented programming aiming at a better 
separation of concerns. Cross-cutting code is captured and localised in 
these aspects. Some of the cross-cutting features which are expressed in 
aspects cannot be woven with other features into the same application 
since two features could be mutually exclusive. With a growing number 
of aspects, manual control of these dependencies becomes error-prone or 
even impossible. We show how assertions can be useful in this respect to 
support the software developer. 



1 Introduction 

Composing systems from individual units has been one of the main research 
goals in the software engineering discipline since its beginning in the 1960s. 
The first approaches concentrating on modules were followed by decomposition 
into objects and / or components [23,4,30,29]. All of these approaches aim at 
managing complexity by dividing a system into smaller pieces. 

Once a systems analyst completes the creative process of dividing the system 
into manageable parts, phases follow where the units are realised and synthesised 
to form the final system. The units themselves are built from scratch or reused if 
they already exist provided they can be used in the chosen technical environment. 

It has become popular to build systems not only for one purpose but to allow 
variability to a certain extent. This is due to the goal to save development and 
maintenance effort and increase software quality by reusing a system multiple 
times even in slightly different contexts. This development is reflected in the 
intensified research in the field of product line architectures (PLA) [27]. In a 
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concrete context the necessary and available units are reused and the points of 
variability are adapted to the particular needs [10,8]. This results in a set of valid 
configurations for one kind of system. The more configurations exist the more 
flexible is the system concerning reuse and adaptation. However, an increase in 
variability and flexibility poses a greater challenge in validating the configuration 
of chosen units. The search for a valid configuration is error-prone if manually 
practised. Therefore, there is a need for computer support. Such computer sup- 
port is also realisable since rules can be used to specify valid configurations in 
advance. 

Aspect-oriented programming (AOP) introduces a new concept called aspects 
[17]. Besides classes, they form an additional type of system unit. Aspects serve 
to localise any cross-cutting code, i.e. code which cannot be encapsulated within 
one class but which is tangled over many classes. The existence of this new type 
of system unit affects the composition validation. Although some composition 
validation mechanisms already exist [3,7], aspects have not been explicitly con- 
sidered. In this paper we introduce a composition validation mechanism which 
in particular concentrates on this new kind of system units. 

In our approach we use the technique of assertions [21]. Assertions are a 
widely accepted programming technique for complex systems to improve their 
correctness. Taking the validity of a configuration as a property, assertions can be 
used to ensure that the chosen aspects and classes fit together. In the following, 
we provide further background information about aspect-oriented programming 
and assertions which is necessary to understand the remainder. A motivating 
example demonstrates the problem we address. Sections 3 to 5 describe our 
different aspect composition validation approaches based on assertions. Section 6 
concludes the paper and identifies directions for future work. 



2 Background and Motivation 

This paper is based on two techniques, i.e. aspect-oriented programming and 
assertions. Here, assertions are used as a means to validate aspect composition. 
In the following, the term “aspect composition” is used for the insertion of a set 
of aspects into one system / class. This set is also called (aspect) configuration. 

Aspect-oriented programming (AOP) [17] introduces a new concept, called 
aspects, in order to improve separation of concerns. It aims at easing program 
development by providing further support for modularisation at both design and 
implementation level. Objects are designed and coded separately from code that 
cross-cuts the objects. The latter is code which implements a non-functional fea- 
ture that cannot be localised by means of object-oriented programming. Code 
for debugging purposes or for object synchronisation, for instance, is tangled 
over all classes whose instances have to be debugged or synchronised, respec- 
tively. Although patterns [12] can help to deal with such cross-cutting code by 
providing a guideline for a good structure, they are not available or suitable for 
all cases and mostly provide only partial solutions to the code tangling problem. 
With AOP, such cross-cutting code is encapsulated into separate constructs or 
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system units, i.e. the aspects. The links between objects and aspects are ex- 
pressed by explicit or implicit join points. An aspect weaver is responsible for 
merging the objects and the aspects. This can be done statically as a phase at 
compile-time or dynamically at run-time [17,15]. For our research, we have used 
Aspect JO. 4beta7 from Xerox PARC [32,18] for all AOP implementations together 
with Java JDK 2. 

Although AOP allows to achieve a better separation of concerns, the applica- 
tion of aspect-oriented programming bears new challenges. This paper addresses 
one of them: aspect composition validation. 

Let us assume we have an application and a set of aspects which can be 
woven with the objects of the application. This set may contain aspects which 
are redundant or even exclusive with respect to other aspects in the set. Al- 
ternatively, weaving one aspect may require another aspect to be woven. For 
example, if there are two different aspects At and A2 containing debugging func- 
tionality, they may be mutually exclusive. At inserts code into the application 
which realises all tracing. Each function call is recorded in a window with graph- 
ical support for user control. As opposed to graphical debug support, A2 realises 
the same tracing functionality as ASCII output. Since both aspects implement 
the same feature (although in different ways) it wouldn’t be reasonable to weave 
both aspects. Depending on the context, the environment or the requirements 
(e.g. towards efficiency), the one or the other is more suitable. 

In [25,28] an example implementation can be found which also reveals this 
aspect composition problem. 

At this point, assertions [21,26] can be used to achieve correctness. The basic 
ideas behind assertions originate in the theory of formal program validation 
pioneered by R.W. Floyd [11] C.A.R. Hoare [14] and E.W. Djikstra [9]. The 
Hoare triples provide a mathematical notation to express correctness formulae. 
Such a correctness formula is an expression of the form: 

{P}A{Q} (1) 

This means that “any execution of A, starting in a state where P holds, will 
terminate in a state where Q holds” [21]. In terms of software, A denotes an 
operation or software element whereas P and Q define the assertions, or pre- 
and postcondition, respectively. By means of assertions it becomes possible to 
state precisely the formal agreement between caller / client and callee / supplier, 
i.e. both, what is expected from and what is guaranteed to the other side. This 
reflects the “Design by Contract” principle [20]. 

For our purposes, the validity of a set of aspects which are woven in A reflects 
an assertion. 

3 Aspect Composition Validation 

Since correctness is always relative ^ [21] we assume to have a certain specifi- 
cation defining (among other things) which features have to be realised in the 

This is also expressed by P in equation (1). 
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application. Therefore, the decision which feature (i.e. here which aspect) should 
be inserted (i.e. woven) is not part of our solution. Moreover, the knowledge 
whether a set of aspects is valid (in this paper also called valid configuration) or 
not, has to be extracted during the analysis phase and captured in the specifi- 
cation (e.g. using finite state automata as outlined in [24]) before assertions can 
be useful to prove this property. 

According to the definition of assertions, we use them to verify that a class or 
an application contains a suitable set of aspects which forms a valid configuration 
(with respect to what is written in the specification). With these assertions, 
the specification is expressed and included in the implementation to assess the 
correctness of aspect composition. In this paper, we concentrate on this issue of 
software correctness and omit all considerations about other properties which 
can be asserted as already known [13]. 

Expressing the part of the specification concerning the aspect composition 
within the code can serve various purposes. These range (similar to other kinds 
of assertions) from treating assertions purely as comments with no effect at 
run-time to checking the assertions during execution. 

Sections 3.1 and 3.2 describe different ways to realise assertions checking for 
a valid aspect configuration. 



3.1 Precondition, Postcondition, or Invariant 

B. Meyer defines an assertion as “an expression involving some entities of the 
software, and stating a property that these entities may satisfy at certain stages 
of software execution” [21]. In our case, the entities of the software involved are 
the classes or methods whereas the stated property is the validity of the aspect 
configuration injected into these classes or methods through the aspect weaver. 

Three possibilities to assert this property can be identified: preconditions, 
postconditions or class invariants [21]. 

First, this property can be asserted as a precondition. The class or method 
starts its work assuming that a valid set of aspects is woven and active. The 
precondition checks whether this assumption is true, i.e. whether the contract 
is fulfilled by the caller and the property to have a valid aspect configuration in 
the callee holds (cf. figure 1 on the left). This may be reasonable if the caller 
changes the aspect configuration which is active in the callee (here we assume 
that the aspects themselves are not changed, i.e. there is no aspect evolution). 
Applying the design by contract principle, the caller ensures that the changed 
aspect configuration is valid for the callee. In the precondition the callee checks 
whether the caller kept the contract. 

Although this is possible, we propose ^ to assert the validity of an active 
aspect configuration in postconditions or class invariants. While pre- and post- 
conditions describe the properties of individual routines, class invariants allow to 
express global properties of the instances of a class which must be preserved by 

^ It should be avoided to assert a property multiple times due to the disadvantages of 
redundant checks and defensive programming [21]. 
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Fig. 1. Different Possibilities to Assert a Valid Aspect Configuration. 



all routines. An invariant for a class is a set of assertions (i.e. invariant clauses) 
that every instance of this class will satisfy at all times when the state is observ- 
able. The crucial point is that with both, postconditions and invariants, it is a 
bug in the callee if the assertion is violated [21]. We believe that in most cases 
it is more reasonable that the decision about which aspect should be active in a 
class C at which times is in the responsibility of C. Thus, in our understanding, 
the callee should decide by itself which aspect instances should be connected to 
this callee. Consequently, as opposed to a precondition realisation, the callee is 
also responsible to ensure and to check that the woven aspects which are ac- 
tive in the callee form a valid configuration. Obviously the same argumentation 
applies symmetrically to the caller which can also be a callee. 

Thus, the remainder of the paper concentrates on postconditions and class 
invariants. It goes without saying that the principles shown are also applicable 
to preconditions. 

3.2 Static or Dynamic 

There are two ways to assess software correctness. The property of a class or 
method to result in a valid aspect configuration can be ensured at run-time 
(dynamically) or alternatively at (or before) weave-time (statically). 

A similar distinction can be identified if the times of aspect configuration 
changes are considered. Either it is possible to add or remove aspect instances to 
or from class instances only statically or even dynamically (cf. table 1; + indicates 
that this combination is possible whereas ++ expresses that this combination is 
preferable provided there is a choice) . 

In the following we show the outlined differences between static and dynamic 
at some Aspect JO. 4beta7 / Java JDK 2 code extracts (cf. figure 2). 

In AspectJ0.4beta7, each aspect has (like classes) its name following the 
keyword aspect and contains the advised methods with their names and the 
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Table 1. Static and Dynamic Change and Validation. 





Change of Aspect Configuration | 




static 


dynamic 


Aspect dynamic 


-f 


+ 


Composition static 


-f-f 


usually 


Validation 




not possible 



class names referring to these methods. Besides method advising it is also 
possible to introduce whole methods. 

Dynamic or Static Change of Aspect Configuration 

At first, the change of the set of aspects which play an active role in a class in- 
stance can be done at weave-time without later changes during execution time. 
This situation is depicted in figure 2 on the left. These static connections between 
an aspect and all instances of a class are expressed with the keyword static in 
AspectJ. Once the weaver injected these static aspects into class staticExample 
at weave-time, the functionality within these aspects will be active in all in- 
stances of StaticExample during the whole execution time. The woven aspects 
augment the class code of StaticExample which impacts all its instances. More- 
over, it is not possible either to add further static aspects nor to remove statically 
woven aspects from one or all instances during run-time. 

As opposed to statically woven aspects, it is also possible to create new as- 
pect instances and connect them to objects during run-time. In figure 2, this 
is shown on the right. The code within the light-grey lined rectangle in the 
DynamicExample class ^ establishes the connection between aspects and class in- 
stance during run-time. The commands used (e.g. addObjectC. . .)) are provided 
by AspectJ0.4beta7. If other AOP environments are used, similar language con- 
structs are necessary. Alternatively, dynamic weaving capabilities can be used if 
available [15]. 

Dynamic or Static Aspect Composition Validation 

Assertions express correctness conditions (here: validity of the aspect configu- 
ration). Assertion rules check whether such a condition is violated. These rules 
can be executed during run-time. If possible, a violation check can also be done 
statically. This requires to have all the necessary information at compile-time or 
weave-time which is true for statically woven aspects. During execution the static 
aspect configuration does not change. Verifying and removing assertions stati- 
cally has the advantage that the overhead of the test during execution is avoided 

® By using the keyword before the aspect weaver injects this advised code at the 
beginning of the method. The keyword after augments the method at the end. 

^ For demonstration purposes we have chosen to present an aspect-directional design 
[16], i.e. the class knows about the aspect but not vice-versa. 
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class MalnStatIc { 

public static void main{String[] args) ( 
-^tatlcExample se = new StatlcExampleQ; 
se.doitO; 

} 

} 



class StatIcExample { 

L ^ public void dolt() { 

^ System. out. println("StatlcExample.doit()"): 
} 

} 



establish Connection 
statically (at Weave-Time) 



* Aspect StaticAspectTest Injects a tracing message 

* before method doit() of class StatIcExample. 



aspect StaticAspectTest { 
static advice void dolt() & StatIcExample { 
before { 

System, out. printin 

("StatIcExample. dolt(): StaticAspectTest"); 

} 

} 






class MainDynamIc { 
public static void main(String[] args) { 
DynamicExample de = new DynamicExample();- 
de.dolt{): 

} 

} 



class DynamicExample { 
public DynamIcExampleO { 

A1 a1 = new A1(); A2 a2 = new A2(); 
a1 .addObject(thlsObject); 
a2.addObject(thlsObject); 

} 

public void doit() { 

System, out. println("DynamicExample.doit()"): 

} 

J 



instantiate 
Aspects and 
establish 
Connection 
dynamically 
(at Run-Time) 



fT" 

' Aspect A1 injects a tracing messages before 
' method dolt() of class DynamicExample. 

’/ 



aspect A1 { 

advice void doit() & DynamicExample { 
before { 

System. out. println("DynamicExample.doit(): A1 "); 

} 

} 

■v}_ 



" Aspect A2 injects a tracing message after 
' method doit() of class DynamicExample. 

’/ 

aspect A2 { 

advice void doit() & DynamicExample { 
after { 

System. out. printin ("DynamicExample. doit(): A2"); 

} 

} 



Fig. 2. Example with Statically or Dynamically Connected Aspects. 



[13]. Thus, although statically connected aspects can be verified at run-time (in 
table 1 depicted with +), assertions execution at compile-time or weave-time is 
to prefer (this is expressed by ++ in table 1). Static assertion violation checking 
is described in section 5. 

Dynamically changing aspect configurations (as depicted on the right in fig- 
ure 2) can hardly be checked at compile-time or weave-time. Which aspect in- 
stance is created and connected to which class instance depends on the dynamic 
control fiow and user input. In this case, the dynamically executed assertions 
are usually unavoidable. The technique for dynamically asserting aspect config- 
urations is described in the next section. 

4 Asserting Dynamically Changing Aspect Configurations 

In this chapter, we concentrate on all aspect composition validation which cannot 
be asserted statically as outlined above. Dynamically created and connected 
aspect instances have to be checked during program execution. These assertions 
can be added into the corresponding class. Since aspects can be used to separate 
assertion functionality from problem domain-related code, it is also possible and 
reasonable to extract the aspect configuration assertion code from the class and 
localise it into a separate assertion aspect. Thus, if a class wants to test whether 
the postcondition (i.e. the property “a valid configuration of active aspects is 
connected”) holds, an assertion aspect has to be woven. As the other aspects. 
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this aspect can be instantiated and connected dynamically or statically. The 
implementation of such a static assertion aspect for DynamicExample of figure 2 
is outlined in figure 3. 



’ Aspect AspectAssert injects assertion functionality after the constructor of 

* DynamIcExampie. Thus it is asserted that the changed aspect configuration is vaiid. 

* / 

aspect AspectAssertion { 
introduction DynamicExampie { 

// -- The facts or knowiedge base and the ruies — 
private static boolean not(String a) { 
if (a .equais(” A3")) return true; // not A3 
if (a .equais(” A4”)) return true; // not A4 
return false ; 

} 



private static boolean xor(Strlng a, String b) { ... } 

// -- Check if any facts and composition ruies are vioiated -- 
private static boolean check(java.awt.List aspect_names) { ... } 



Knowledge Base and 
Rules defining valid 
Aspect Configuration 



public boolean assert_composition() { 

// Obtain aspect references of aspects which are active: 
java. util. Vector v = thisObject.getAspects(); 

// Derive "aspect_names" list with the aspect names from v 

boolean b = check(aspect_names); 
return (b ) ; 

} 

} 

// -- Postcondition is injected by augmenting the constructor -- 
static advice void new(..) & DynamicExampie { 
after { 

if (assert_composition{)) throw new PostConditionViolation(); 

} catch (Exception e) { Syste m.out.println(e .toStringO); } 

} 



Check active 
Configuration 
against 

Knowledge Base 
and Rules 



Assertion 



Fig. 3. Assertion Aspect for Dynamic Testing. 



Note that with the introduction of these new assertion aspects, these as- 
pects themselves may be ensured by higher-level assertion aspects. This situation 
can be compared to the abstraction hierarchy in object-orientation (“instance”, 
“class” and “meta-class”). The crucial question is how many abstraction levels 
are reasonable. These considerations are beyond the scope of this paper [31]. 

According to figure 3, the knowledge base and rules part expresses the knowl- 
edge contained in the specification. It defines all valid aspect configurations. The 
actual configuration is determined with the AspectJ command 
thisObject.getAspectsO during run-time which returns all active aspect in- 
stances connected to the class instance. Such a possibility to determine all woven 
and active aspect instances at run-time is crucial to their dynamic assertion. This 
determined set of active aspect instances is then checked against the knowledge 
base and rules. If this check proves that the postcondition is true, an invalid 
aspect configuration is detected and an exception is raised (cf. figure 3). 

An interesting part is the one expressing the knowledge base and the rules 
of a valid aspect configuration. Some basic types of rules can be identified which 
allow to express the relevant dependencies between the aspects and between the 
aspects and a certain class instance (with respect to the specification). For our 
various example implementations the following rules proved to be sufficient: 
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— not Al: A not-clause expresses that the aspect with name At is not allowed 
to be woven into the class instance. 

— requires Al: Note that this term differs from require clauses sometimes used 
to express assertions (e.g. in the programming language Eiffel [19]). Here, 
it is meant that weaving aspect Al into the class instance is mandatory. 
This rule may also have multiple arguments (e.g. requires Al A2 A3) which 
indicates that at least one of these aspects is mandatory (in the example 
either Al or A2 or A3 is mandatory). 

— xor Al A2: This expression indicates that either Al or A2 may be woven into 
a class instance but never both of them. 

— cuid Al A2: An and-clause expresses that Al and A2 have to be woven together 
into the same class instance. 

In figure 3 the methods not and xor outline a possible implementation of 
two types of rules in AspectJ0.4beta7 / Java JDK 2. The check(. . .) method 
describes the application of these rules to a set of aspects. 

Although these dependency rules are expressed in AspectJ0.4beta7 / Java 
syntax in the presented implementation a multi-paradigm approach [6,5] would 
be suitable here. With a logic programming language (e.g. Prolog) the imple- 
mentation of the knowledge or dependency rules contained in the specification 
would lead to improved understandability. Generally speaking, a domain-specific 
language [7] based on the predicate calculus would improve the implementation. 
Alternatively, a domain-specific language based on finite state automata can be 
used to express these dependencies as described in [24]. 

An assertion aspect for a certain software system may be also generated from 
a file containing the knowledge base and rules [28]. 

5 Asserting Static Aspect Configurations 

While the validation of the dynamic aspects results in performance penalties, 
these can be avoided for static aspects since the configuration can be assured 
before the execution (cf. table 1). It is a common technique to remove assertions 
at compile-time provided they can be checked in a static analysis [13]. 

The principle of such static validation procedure is as follows: Assuming we 
have a class implementation including static assertion clauses to assure a valid 
aspect configuration. Then, the compiler, weaver or any tool operating before 
execution time can examine this class implementation. The statically known 
information about what has to be asserted can be extracted from the code and 
checked before run-time. The dynamic tests during execution can be avoided 
since the already statically checked assertions can be eliminated. 

Based on this observation, we built the ‘^Aspect Composition Validatioii' 
tool (cf. figure 4) in order to automate the static verification. The tool realises 
a slightly different verification procedure compared to the principle described 
above. We chose not to inject assertion code into the classes. This eliminates 
the need to remove this code before execution. The developer decides by menu 
which classes the tool has to assert. The assertion itself (i.e. the property that is 
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^Aspect Composition Validation 



File Aspects Aspect Dependency Rules Help 



Aspect File: 



ISiqiply.java 

iPane.jsva 



Aspect Dependency Rules: 



/* GeneratedbyAspectJvenion0.4beta7 *f 

class Action extends java.lang. Object { 
protected final void Action$dioit$signatureO { 
java.lang.System.outprintln('’Action: doitO ”); 

) 

public void doitO ( 

{ 

I* 

* Generated by ajc 

* This implements the before advice ... 

* crosscut action from the aspect TraceAspect2. (TraceAspect2.ja 

*1 

{ 

java.utiLEnumeration _enumeration = _aspects.elementsO> 
while (_enumeration.hasMDreElementsO) { 
java.lanf.Object _thisAspect = _enumeration.nextneinentO; 
if(_fiiisAspectinstanceofTraceAspect2) { 
final TraceAspect2 thisAspect = (TraceAspect2)_thisAspect; 



J 






violated Dependency Rules: 



1 and TraceAspectZ TraceA5pect3 




- 


2 requires TraceAspecIl 






3 xor TraceAspectl TraceAspcct2 






4 and TraceAspectS I^rseAspect 






5 requires TraceAspectd 


_l 


J 



3 xor TraceAspectl TraceAspect2 

5 requires TiaceAspectd 

6 not ParseAspect 



AspectList: 



TraceAspectl 
TraceAspectl 
Trace Aspects 
PaneAspect 
TraceAspect2 



clr Aspect 



clr Aspect list 
clr Dep List 



Fig. 4. Aspect Composition Validation Tool. 



to be asserted) is obvious: the aspects connected to the class have to be checked 
according to aspect dependency rules derived from the specification. 

The tool can be used to ensure the property of a valid static aspect config- 
uration within a specific class. Such a class-wide validation corresponds to the 
concept of class invariants described in [21]. Moreover, the tool is also able to 
verify the aspect configuration of a set of classes. Such a set may be a pack- 
age, a component or a subsystem. Therefore, this is an extension of B. Meyer’s 
understanding of assertions (which is limited to pre-, postconditions and class 
invariants) to more than a single class, i.e. to more coarse-grained building blocks 
of the system. In the tool, the single file or the set of files containing the class(es) 
to be validated are listed in the chosen Files list with their aspects displayed in 
the field Aspect List as shown in figure 4. Additionally, the highlighted file in 
the file list is presented in the Aspect File text area. 

The tool verifies the aspect configuration (consisting of all the aspects which 
are included in the listed files on the left) according to the aspect dependency 
rules (initiated by pressing the validate button). Violated aspect dependency 
rules are displayed in the list named violated Dependency Rules. The aspect 
dependency rules expressing the invariants of aspect combination are the same as 
those described in section 4 (not, requires, and and xor). The software developer 
can insert or change these aspect rules directly in the Aspect Rules text area. 
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Alternatively, they can be read from a user-defined file containing these rules 
written in the domain-specific language. 

The Aspect Composition Validation tool extracts the static aspects by pars- 
ing the files containing the already woven source. AspectJ0.4beta7 marks all 
woven code sections with comments. Alternatively, the woven aspects could be 
derived from specific documentation files (with extension . corr) provided by 
Aspect JO. 4beta7. This file documents which aspect is woven in which class or 
method. The woven aspect set could also be obtained from the original source 
files directly. 



6 Conclusion 

Although aspect-oriented programming can improve software due to a better 
separation of concerns, the software developer is faced with new challenges. One 
of them is the aspect composition validation which is not yet examined suffi- 
ciently in AOP and therefore is addressed in this paper. 

Both the dependencies between the aspects woven with a class instance and 
the dependencies between these woven aspects and the class instance itself have 
to be identified in a specification. On this basis we demonstrate how to use 
assertions to ensure the correctness of these dependencies with respect to the 
specification. Faulty aspect configurations (e.g. if the set of woven aspects em- 
braces aspects which are contradictory) can be detected using assertions similar 
to other bugs. Dynamically changing aspect configurations are checked at run- 
time. Due to the performance penalty in case of dynamic tests, we also presented 
a static analysis which is preferable in case of static aspect configurations. Al- 
though the feasibility of the principles and concepts in this paper is shown with 
Aspect JO. 4beta7 and Java JDK 2 implementations the approach is indepen- 
dent of the concrete technology. For instance, using composition filters [2], [1] to 
achieve a better separation of concerns also leads to the problem that both the 
order of the filters and the filters of the different objects within one application 
have to fit semantically. Assertions can be used similarly in this case. 

Our further research will concentrate on transferring these concepts to com- 
ponent or object composition validation with respect to already existing support 
in this domain [3] . There is a growing need for computer support in the field of 
composing systems of reusable parts which are stored in repositories. As with 
aspects, there might be multifarious dependencies between these parts. Another 
field is the graphical representation of the dependencies and of all violated de- 
pendency rules within a certain application. Besides this graphical feature, a 
tool environment can include further support for the software developer. The 
description of a valid configuration, i.e. of the knowledge base and the rules ac- 
cording to the specification, should be separated. Further investigation in the 
sense of domain engineering is necessary there. Moreover, a partial automatic 
generation of assertion aspects from such separated descriptions is possible and 
will be another research activity in our future work. 
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Abstract. The increasing number of available software components and 
competing interoperability standards render the selection, composition, 
and configuration of software components increasingly complex. In order 
to support the domain expert in these processes comprehensive, com- 
parable, and sufficiently abstract component descriptions are required. 
In this paper, we abstract from the specifics of the component models 
COM, EJB, and CCM, and propose a unifying component description 
language for integrated descriptions of structure and behaviour of soft- 
ware components and component-based software systems. 



1 Introduction 

The notion of component-based software development denotes an approach pur- 
sued in every mature engineering discipline: systems (here: software) shall be 
developed by reusing and combining well-tried methods and tested products 
(here: software components) [8]. Szyperski defines a software component (fur- 
ther on only referred to as component) as “[...] a unit of composition with con- 
tractually specified interfaces and explicit context dependencies only. A software 
component can be deployed independently and is subject to composition by third 
parties” [25]. In this context a contract represents an agreement between a client 
and a provider concerning a set of services. It specifies how to use these services 
and imposes requirements on their implementation. 

Within the field of component-based software development two kinds of stan- 
dardization efforts can be distinguished: the technical standardization of com- 
ponent interfaces (e. g., by component models such as Component Object Model 
(COM), Enterprise JavaBeans (EJB), and CORBA Component Model (CCM)) 
on the one hand provides for improved exchangeability of components on a tech- 
nical level, competing interoperability efforts (e. g., by OAGIS, OMG, Roset- 
taNet) on the other define a multitude of domain interfaces which may be sup- 
ported by software components. These standardization efforts intend to permit 
the combination of “best of breed” solutions, however, they overwhelm the do- 
main expert with the resulting variability. Therefore, the selection, composition. 
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and configuration of components and component-based application systems in- 
creasingly require suitable methods and tools supporting the domain expert. 
A comparable problem can be found in the field of standard software. Current 
standard software products excel by their high configurability and are therefore 
applicable in many different contexts [1]. In order to render this richness of vari- 
ants more comprehensible and manageable for domain experts, standard ERP 
software products are described by software reference models (cf. [12]). Soft- 
ware reference models illustrate the business processes, organizational and data 
structures supported by a particular software product as well as its configuration 
alternatives on a level of abstraction which is suitable for domain experts. 

We suggest to transfer the idea of software reference models to component- 
based software development in order to facilitate the sound selection, composi- 
tion, and configuration of components and component-based software systems. 
Component-oriented software reference models are to support the domain ex- 
pert in dealing with component software by abstracting from technical informa- 
tion and focusing on domain-related information instead. We envisage deriving 
component-oriented reference models in the shape of, e. g., business process mod- 
els, message sequence charts, and controlled language documents from uniform, 
comprehensive, and sufficiently formal component descriptions. The specification 
languages employed by current component models are improper as a basis for 
such component-oriented software reference models. On the one hand, they align 
themselves too much with the specifics of the respective component model, and 
on the other, they are restricted to the signature level of interface descriptions. 
Important information on interaction protocols and terminological semantics 
of components may not be formulated using these languages. In this paper we 
therefore propose a language CDL for the integrated description of structure and 
behaviour of components and component-based software systems. CDL provides 
a comprehensive, unified view on the standard component models COM, EJB, 
and CCM by abstracting from technical specifics, yet ensuring the ability to ex- 
press the most important concepts. In addition to a component’s signature, CDL 
descriptions may also comprise its formal semantics in the shape of interaction 
protocols. 

The structure of our paper is as follows. Section 2 outlines the most important 
concepts of the component models COM, EJB, and CCM, and presents the CDL 
component model which is intended to serve as a unified view on COM, EJB, 
and CCM components. In Sect. 3 the description language CDL is introduced 
using a simple example. Section 4 gives an outlook on future enhancements of 
CDL and presents related research. Finally, Sect. 5 summarizes our contribution 
and relates it to complementary research interests. 

2 A Unified View on Standard Component Models 

In this section we first delineate the currently most popular component models 
COM, EJB, and CCM. We concentrate on the relationships between contracts, 
implementations and runtime objects of their components. A more detailed com- 
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parison of COM+, COM’s current successor, EJB, and CCM can be found in 
[24]. Thereafter, we present the CDL component model which is intended to 
serve as a unified, abstract view on component software. A brief comparison of 
the CDL component model with the component models COM, EJB, and CCM 
is presented in a slightly extended version of this paper [26]. 

2.1 Component Object Model (COM) 

In Microsoft’s Component Object Model (COM) the services provided by a com- 
ponent are specified by interfaces which represent sets of methods. COM classes 
are named implementations of at least one interface, its instances are referred 
to as COM objects. COM objects are created by means of class factories. A 
COM component packages one or more COM classes and constitutes the unit 
deployable in software development projects. COM does not support inheri- 
tance between components [6]. Since COM classes do not explicitly publish the 
interfaces they support, it is in general necessary to validate a COM class’s suit- 
ability for a given application context by successively checking the availability of 
required interfaces {interface negotiation). In order to avoid this often large num- 
ber of checks, COM provides the concept of categories. Categories denote sets 
of interfaces and make the notion of contracts explicit [25]. Figure 1 shows the 
basic concepts of COM and the relationships between. The figure insinuates an 
object-oriented implementation of COM classes, however, a non-object-oriented 
implementation is also conceivable. 
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Fig. 1. Component Object Model (COM) 



An interesting feature of COM is the fact that a COM class may support 
different versions of an interface simultaneously. COM prohibits changes to in- 
terfaces once they are specified. Thereby COM avoids the syntactic and semantic 
fragile base class problem which pertains to the question whether a base class 
(a component) can evolve without breaking independently developed subclasses 
(dependent components) [25]. In order to be able to provide new or advanced 
functionality nevertheless, new versions of a COM class may offer additional 
interfaces comprising new or updated versions of existing functionality. 
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2.2 Enterprise JavaBeans (EJB) 

Sun Microsystems’s Enterprise JavaBeans (EJB) is oriented towards the devel- 
opment of distributed server-side applications [14]. A server provides an envi- 
ronment for the execution of applications. Within a server, eontainers manage 
the components of the EJB component model: the enterprise beans. Contain- 
ers provide interfaces for the platform independent development of both clients 
and enterprise beans. These interfaces are specified by means of two contracts. 
The client contract constitutes the contract between client and container. It 
governs a client’s access to the enterprise beans deployed within the container. 
Every enterprise bean defines a home interface which offers methods for cre- 
ating, finding, and destroying instances of the respective bean, and a remote 
interface providing application specific methods of the bean. The component 
contract specifies the interface between a container and the enterprise beans it 
contains. It requires a container to implement both an enterprise bean’s home 
and remote interface. The implementation of a remote interface thereby has to 
delegate method invocations issued by a client to the ejb-class coming with the 
enterprise bean and providing the implementation of the bean’s functionality. In- 
stances of the implementations of home and remote interface are referred to as 
EJBHome and EJB Object, respectively [14]. EJB does not support inheritance 
between enterprise beans. Figure 2 depicts the views “contract” , “implementa- 
tion” and “runtime objects” on EJB. 
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Fig. 2. Enterprise JavaBeans (EJB) 



2.3 CORBA Component Model (CCM) 

The specification of the Object Management Group’s (OMG) CORBA Compo- 
nent Model (CCM) is not yet finished. The following survey on the GORBA 
Component Model grounds on a joint revised submission to the OMG made by 
a number of renowned software development companies [20]. 

The CORBA Component Model combines concepts from COM and EJB and 
thus represents in a way a “best of both worlds” component model. A component 
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type denotes a specification of a component’s features and may encapsulate sev- 
eral implementations (e. g., for different platforms). Similar to EJB, instances of 
component types, the CORE A components (CORCs), are managed by homes. 
A CORBA component has a single distinguished reference whose interface con- 
forms to the component’s equivalent interface. This interface publishes the com- 
ponent’s features (referred to as ports). The kinds of ports supported by CCM 
are facets, receptacles, event sources and sinks, and attributes. Similar to non- 
standard interfaces in COM, facets represent the functional application interfaces 
of a CORBA component. A CORBA component may explicitly support addi- 
tional interfaces next to its facets. Receptacles represent an abstraction for the 
creation and management of typed connections between a CORBA component 
and an object (e. g., a reference to a facet or the home of another CORBA compo- 
nent) . Event sources push events to consumer interfaces provided by event sinks. 
Attributes serve the configuration of components. Figure 3 illustrates the most 
important concepts of the CORBA Component Model in the acquainted form. 
For the sake of clarity the possibility of single inheritance between component 
types is not shown in this figure. 
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Fig. 3. CORBA Component Model (CCM) 



2.4 The CDL Component Model 

The CDL component model is intended to provide a unified abstraction from 
the component models introduced in the preceding paragraphs. According to 
Szyperski components in general may provide two different kinds of interfaces. 
Direct interfaces correspond to procedural interfaces known from classical soft- 
ware libraries. Indirect interfaces are interfaces of objects provided by compo- 
nents [25]. Analogously to these types of interfaces, the CDL component model 
comprises components which may be called using a direct interface, and classes 
which are packaged by components and whose instances - the business ob- 
jects - may be called by means of an indirect interface. The CDL component 
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model distinguishes between atomic, i. e. non-composite components, and com- 
plex components, i. e. composites of atomic and/or complex components. Atomic 
components can be further subdivided into closed components and eomponent 
frameworks. While closed components are not extensible, component frameworks 
represent the basis for the construction of complex components. They provide 
extension points ( “hot spots” ) for their connection with other components. 

An important issue in a component-based software system is the substitutabil- 
ity of its components. If, e.g., a company wanted to substitute a more modern 
production planning component for the one contained in its business application 
system, the new component is required to show the structure and behaviour 
expected by the application system. Within the CDL component model, the 
notion of substitutability is addressed by contracts. Component frameworks use 
contracts to specify types of components which may be “plugged into” their 
extension points. Similar to components comprising classes, contracts may con- 
tain objecttypes. An objecttype specifies a type of business object. A contract 
may be implemented by arbitrary components, and conversely a component may 
support an arbitrary number of contracts. Each of a component’s classes may 
realize one or more objecttypes of a contract. Figure 4 illustrates the concepts 
of the CDL component model. 
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Fig. 4. CDL Component Model 



3 Comprehensive Description of Software Components 

In this section we introduce CDL (Component Description Language), a revised 
version of the SCDL (Simple Component Description Language) proposed in [23] . 
CDL grounds on the CDL component model outlined in the preceding section. 



3.1 The Component Description Language 

The primary goal of CDL is to provide a unified foundation for the description 
of components of different component models. CDL descriptions are intended to 
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serve as a basis for the derivation of domain-oriented models like, e.g., event- 
driven process chains (EPCs), message sequence charts or controlled language 
documents, which facilitate the assessment of software components especially 
for domain experts. The orientation of CDL towards domain aspects is under- 
pinned by the idea to explicitly support underspecifications of components and 
contracts. Such underspecifications might result from incomplete descriptions of 
method and procedure parameters as well as non-deterministic choices. An ex- 
ample of underspecifications are the EPCs of the R/3 reference model [12]. These 
render the behaviour of the R/3 system only fragmentarily. However, they serve 
the purpose of supplying users and consultants with a sufficient overview over 
the functionality of the R/3 system. 

The description languages of the component models COM, EJB, and CCM 
show various deficiencies and are therefore not suitable as a foundation for 
domain-oriented descriptions of components and component-based software sys- 
tems. COM IDL [2] is constrained to the description of collections of methods. 
EJB’s deployment descriptors [14] or CCM’s component assembly descriptors 
[20] permit the formulation of structural information like, e.g., on the archi- 
tecture of component-based application systems. However, both EJB and CCM 
descriptions do not explicitly specify the methods comprised by a component or 
business object interface. This information can only be retrieved indirectly by 
means of introspection (in this case the component has to be present) or access 
to an interface repository, respectively. From our point of view the gravest defi- 
ciency, however, is the fact that non of these description languages regards inter- 
action protocols, i. e. they are restricted to the signature level. The specification 
of a component’s behaviour in the shape of the interactions the component may 
engage in is indispensable for semantically rich, domain-oriented descriptions. 
For this reason CDL comprises constructs for the description of both structural 
and dynamic aspects of components and component-based software systems. 
According to Harel and Gery a minimal set of languages for the description of 
object-oriented systems consists of object-model diagrams and statecharts [9]. 
In addition to statecharts various other approaches to behavioural descriptions 
do exist. Pre- and post-conditions constitute a solid formal foundation, however, 
they are not operational and do therefore not represent an adequate basis for 
domain-oriented models [27]. Process terms based on process calculi like CCS 
[15], CSP [11] and the 7r-calculus [17] permit an operational representation of 
processes and interactions. Another approach to behaviour descriptions grounds 
on the insight that different process terms may denote the same set of commu- 
nication sequences (traces). Thace specifications therefore abstract from process 
terms and describe behaviour declaratively as sets of traces, thereby losing the 
operational representation [19]. CDL grounds on process terms on the basis of 
the polyadic 7r-calculus and concepts of statecharts for the specification of com- 
ponent behaviour as well as object-model diagrams for the description of struc- 
tural properties. The polyadic 7r-calculus [16] represents an extended version of 
the original Tr-calculus. It permits tuples and compound types to be sent along 
communication channels. 
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CDL distinguishes between component and contract descriptions. Apart from 
underspecification, a component description is intended to define an upper bound 
to the set of a component’s possible interaction sequences. This semantics is in- 
teresting for safety reasons: a component does not engage in more interactions 
than asserted by its description. Accordingly, a contract description specifies a 
lower bound to the set of expected interaction sequences (liveness) . This perspec- 
tive guarantees that a component which supports a particular contract provides 
the services required. Ebert and Engels coined the terms observable behaviour 
and invoeable behaviour for these semantics of behaviour descriptions [5]. Analo- 
gous formulations can also be found in the field of process terms: a process “may 
engage” or “must engage” in a set of traces, respectively [19]. 



Describing Atomic Components. In this section we introduce those aspects 
of the component description language CDL which are relevant to the description 
of atomic, closed components and corresponding contracts. Figure 5 shows a 
description of an atomic, closed component AccountComponent which may be 
employed for the management of accounts. 

Component descriptions begin with the keyword component. They may con- 
tain descriptions of arbitrary classes which are marked by the keyword class and 
specify implemented types of the component’s business objects. The component 
AccountComponent in Fig. 5 comprises only one class named Account. A com- 
ponent’s direct interface is represented by procedures. Within our example, the 
direct interface of AccountComponent consists of a procedure openAccountO 
for opening a new account. The indirect interface of a component is represented 
by its classes and their methods. The class Account defines methods deposit () 
and withdrawO for depositing money in an account and withdrawing it, and 
close () for closing an account. Parameters of procedures and methods are di- 
rected (in, out, inout) and typed using the classes, contracts, and objecttypes 
known within a component’s scope as well as base types like int, float and 
string. 

In addition to these structural description elements, a component description 
may also encompass descriptions of a component’s and its classes’s behaviour. 
For this purpose we employ the idea underlying communication within the tt- 
calculus: a procedure or method identifier followed by a “!” indicates the sending 
of a call to the procedure or method. The readiness for the receiption of such 
a call is specified by a “?” following the identifier. A procedure or method is 
executed, if a send action can be matched with a corresponding receive action. 
The description of a component’s or class’s behaviour using CDL is based on 
the idea to separate the specification of the active behaviour, i. e. the procedure 
and method calls issued, from the specification of the passive behaviour, i. e. the 
readiness to receive such calls (cf. [19]). The active behaviour of a component or 
class is specified in procedure and method bodies, while the passive behaviour 
is formulated in specific behaviour descriptions. The latter consist of states and 
transitions, whereby each behaviour description comprises two standard states: 
initially, an instance of a component or a class is in the state created. This state 
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component AccountComponent { 
class Account { 

deposit (in sum: float) { 

return; /* increase of balance not described */ 

}; 

withdraw(in sum: float) { 

return; /* decrease of balance not described */ 

}; 

closeO { 

return; /* closing of account not described */ 

}; 

createdO 
initializedO 

balance (value : float) 



closedO 

}; 

openAccount(out account: AccountComponent. Account) { 
netr(AccountComponent .Account account) ; 
return(account) ; 



= initializedO ; 

= deposit? (sum) .balance (sum) 

+ close?() .closedO ; 

= deposit? (sum) . balance (value+sum) 

+ on withdraw? (sum) 
if (sum <= value) 

do balance (value-sum) 

+ on close?() 

if (value == 0) 
do closedO ; 

= 0 ; 



createdO = initializedO ; 

initializedO = openAccount? (account) . initializedO ; 



Fig. 5. Component AccountComponent 



can be associated with an initialization script, i. e. a sequence of procedure or 
method calls. After this initialization the component or class instance progresses 
to the initialized state which represents its effective start state. Behaviour 
descriptions associated with this or a subsequent state may only describe the 
readiness to receive procedure or method calls, i. e. its passive behaviour. 

Both the component AccountComponent and the class Account do not re- 
quire any initialization. For this reason they directly progress from state created 
to the successor state initialized. AccountComponent is then ready to ac- 
cept calls to the procedure openAccount (). Whenever this procedure is called, a 
new instance of class Account is created and returned using the out parameter 
account. Subsequently, AccountComponent returns to the state initialized. 
The new instance of class Account is initially prepared to accept deposits. If 
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a deposit of amount sum is made by issuing a call to deposit (), the instance 
progresses to the state balance (sum), i. e. states may have parameters of the 
base types mentioned earlier. As an alternative to making deposits the Account 
instance may be closed again. This choice is global non- deterministic, i. e. it de- 
pends on the interaction behaviour of the clients, and is specified by the operator 
+. After calling the method close () the instance progresses to the state closed 
which only comprises the empty process term () and therefore rules out any fur- 
ther interaction with the Account instance. Associated with the state balance 
we describe the conditional readiness of an Account instance to accept calls to 
its method withdraw (). The amount of money to be withdrawn is limited by 
the current balance. The semantics of the construct on <call> if <condition> 
do <statechange> is derived from the idea of call events employed within UML 
state machines [21]: in case of a call to the procedure or method specified by 
<call> the condition <condition> is checked. If this condition holds, the call is 
executed, otherwise it is rejected and an exception is generated. Concerning the 
communication of exceptions, we employ the convention that every call returns 
a truth value (true, if the call was successful, false, otherwise). The calling com- 
ponent or class instance may define reactions to these truth values by means of 
an if-then statement. 

Descriptions of contracts and components do hardly differ syntactically: 
merely the keywords component and class for the description of components 
need to be replaced by the keywords contract and objecttype for the descrip- 
tion of contracts. 



Describing Complex Components. In order to present the constructs pro- 
vided by CDL for the description of complex components we enhance our exam- 
ple by introducing a component framework which will be extended by a compo- 
nent. Figure 6 shows the description of a component framework BankComponent 
which realizes a fragment of a bank’s functionality. The BankComponent com- 
prises a class Bank, whose instances represent individual banks. The direct inter- 
face of BankComponent permits the creation of an arbitrary number of Banks. 
The interface of the class Bank defines methods openAccount () for opening an 
account and transfer () for transferring money between two accounts. The de- 
scription of the method transfer!) refers to the truth values returned by each 
method call (cf. Sect. 3.1): in case the withdrawal required for the transfer fails, 
no subsequent deposit will be made. 

Component frameworks differ from closed components in being extensible 
by other components. In CDL such extension points of component framework 
descriptions are specified using the keyword slot. Slots are typed with contracts 
which define requirements to compatible components. The description of Bank- 
Component, e.g., comprises a slot named accComp into which components com- 
plying with the contract AccountContract depicted in Fig. 7 may be “plugged” . 
The contract description AccountContract specifies a set of components which 
may be employed by the BankComponent for account management. It differs from 
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component BankComponent { 

slot AccountContract accComp; 

class Bank { 

openAccount (out account: AccountContract. CAccount) { 
accComp. openAccount ! (account) ; 
return account; 

}; 

transfer (in source: AccountContract . CAccount , 

in dest: AccountContract . CAccount , in sum: float) { 
if source .withdraw! (sum) then 
dest . deposit I (sum) ; 
return; 

}; 



>: 



createdO = initialized () ; 
initializedO = ( openAccount? (account) 

+ transfer? (source, dest. 



sum) ) .initializedO ; 



createBank(out bank: BankComponent . Bank) { 
new (BankComponent . Bank bank) ; 
return (bank) ; 

>: 



createdO = initializedO; 

initializedO = createBank? (bank) .initializedO 

>; 

Fig. 6. Component Framework BankComponent 



the component description AccountComponent shown in Fig. 5 in particular in 
not requiring the opportunity to close accounts. 

A complex component represents a configuration of a component framework 
where every slot is filled by a component complying with the respective con- 
tract. It is defined by specifying a component framework and a set of com- 
ponents to be “plugged into” the framework’s slots. If one regards component 
frameworks as templates, a complex component represents an instance of such 
a template. Figure 8 depicts the description of a complex component XYBank 
which is composed from the components BankComponent and AccountCompo- 
nent. The composition of BankComponent and AccountComponent represents a 
valid configuration, since AccountComponent complies with AccountContract. 
The description of AccountComponent encompasses every trace specified by the 
contract description AccountContract and defines the same set of possible fail- 
ures, i. e. rejected traces (cf. [18]). The fact that an AccountComponent can be 
closed is irrelevant from the BankComponent ’s perspective. If the compliance of 
AccountComponent with AccountContract had been known at the component 
description’s development time, this could have been explicitly expressed in CDL 
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contract AccountContract { 
objecttype CAccount { 

deposit (in sum: float) { 

return; /* increase of balance not described */ 

>: 

withdraw(in sum: float) { 

return; /* decrease of balance not described */ 

}; 



created () 
initializedO 
balance (value : float) 






initializedO ; 
deposit? (sum) .balance (sum) ; 
deposit? (sum) . balance (value+sum) 
+ on withdraw? (sum) 
if (sum <= value) 

do balance (value-sum) ; 



openAccount(out account: AccountContract. CAccount) { 
new(AccountContract. CAccount account) ; 
return(account) ; 

>; 



createdO = initializedO; 

initializedO = openAccount? (account) .initializedO ; 



Fig. 7. Contract AccountContract 



complex XYBank = BankComponent(AccountComponent) ; 

Fig. 8. Complex Component XYBank 



by a supports statement (cf. Fig. 9). Such an explicit specification, however, is 
not a prerequisite for “plugging” a component into an extension point of a com- 
ponent framework. 



component AccountComponent supports AccountContract f 
class Account { 

>; 



Fig. 9. Explicit Support of Contract 



To summarize, CDL permits the integrated description of structure and be- 
haviour of components and component-based software systems. Moreover, CDL 
explicitly supports the composition of components by its ability to describe com- 
ponent frameworks and complex components. 
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4 Future and Related Work 

Clearly, the ability to map the essential concepts of the standard component 
models COM, EJB, and CCM to CDL does not suffice for a comprehensive 
component description language. In its current shape CDL only supports the 
specification of functional properties of components and component-based soft- 
ware systems. In this regard we intend to extend the description of exceptions 
thrown by failed method calls by typed exceptions. Moreover, the restriction 
of supported events to call events will be revised. From our rather business 
process-oriented perspective, however, support for additional types of events is of 
subordinate importance. Inheritance between components will not be supported 
in CDL since we do not expect inheritance to play a major role in component- 
based software development. In the context of independent actors developing and 
composing components we consider the concepts of structural and behavioural 
subtyping to be more important. Regarding non-functional properties of com- 
ponents we intend to enhance CDL by adding constructs for the description of 
properties such as time and space requirements, and service levels. In its current 
version CDL is restricted to information on the level of signatures and interac- 
tion protocols. In order to remedy this deficiency, we are eventually interested 
in questions regarding the semantics of the terminology employed within CDL 
component descriptions (semantic level of interoperability). 

A number of different approaches to the description of components and 
component-based software systems do exist, some of which have influenced the 
design of CDL. Wing and Zaremski employ pre- and post-conditions for the 
specification of components. They define relaxed notions of generalization, spe- 
cialization, and substitutability of components [27]. The architecture description 
language Darwin is based on the 7r-calculus. In Darwin an architecture is made 
up of components which are connected through the services they require and pro- 
vide [13]. Canal et al. extend CORE A interfaces by adding behavioural descrip- 
tions based on the 7r-calculus. They aim at questions related to substitutability 
and compatibility of COREA objects [3]. The information system specification 
language TROLL combines both declarative and operational concepts for the 
specification of object systems. The behaviour of objects and object systems is 
described using temporal logic and some dialect of CSP [10]. BOCA CDL is the 
specification language of the Business Object Component Architeeture. EOCA 
CDL is a declarative language for the specification of enterprise object inter- 
faces, relationships between enterprise objects, and behaviour of communities of 
such objects [4]. CSB-OZ is an integrated formal method which combines the 
state-oriented method Object-Z with the process algebra CSP. A CSP-OZ spec- 
ification describes a system as a collection of interacting objects, each of which 
is described regarding structural and dynamic aspects [7]. 

5 Concluding Remarks 

Starting from an outline of the component models Component Object Model 
(COM), Enterprise JavaBeans (EJB), and CORBA Component Model (CCM) 
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we have proposed a unified abstraction from component software in the shape 
of the CDL component model. On the basis of the CDL component model we 
have proposed CDL, a language for the integrated description of structure and 
behaviour of components. CDL is a revised version of SCDL, which has been for- 
mally defined by Ritter by means of a translation to PICT [22], an asynchronous 
variant of the 7r-calculus [23]. We have introduced CDL by means of a simple 
example indicating how to describe components and component-based software 
systems using CDL. 

Our work concerning CDL is strongly linked to the research project KOSO- 
BAR (Component-Based Software Development Based on Reference Models) 
which is currently conducted at OFFIS. Within the scope of this project we are 
concerned with the development of tools and methods for the distributed devel- 
opment of component-based software. Thereby we assume that corresponding 
software development processes ground on the interaction between actors like 
component developers, application architects, consultants, and users. Current 
work within this project deals with mapping the CDL component model to 
the Unified Modeling Language (UML) [21] and developing a configuration de- 
scription language for component-based software systems. A proposal for such 
a language on the basis of description logics can be found in [23] . Future work 
besides the enhancements of CDL outlined in Sect. 4 will be concerned with the 
derivation of domain-oriented models like, e. g., EPCs, message sequence charts, 
interaction diagrams or controlled language documents from CDL descriptions. 
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Abstract. Component-based development of language tools stands in need of 
meta-tool support. This support can be offered by generation of code - libraries 
or full-fledged components - from syntax definitions. We develop a comprehen- 
sive architecture for such syntax-driven meta-tooling in which grammars serve 
as contracts between components. This architecture addresses exchange and pro- 
cessing both of full parse trees and of abstract syntax trees, and it caters for the 
integration of generated parse and pretty-print components with tree processing 
components. 

We discuss an instantiation of the architecture for the syntax definition formal- 
ism Sdf, integrating both existing and newly developed meta-tools that support 
Sdf. The ATerm format is adopted as exchange format. This instantiation gives 
special attention to adaptability, scalability, reusability, and maintainability issues 
surrounding language tool development. 



1 Introduction 

A need exists for meta-tools supporting component-based construction of language 
tools. Language-oriented software engineering areas such as development of domain- 
specific languages (DSLs), language engineering, and automatic software renovation 
(ASR) pose challenges to tool-developers with respect to adaptability, scalability, and 
maintainability of the tool development process. These challenges call for methods and 
tools that facilitate reuse. One such method is component-based construction of lan- 
guage tools, and this method needs to be supported by appropriate meta-tooling to be 
viable. 

Component-based construction of language tools can be supported by meta-tools 
that generate code - subroutine libraries or full-fledged components - from syntax def- 
initions. Figure 1 shows a global architecture for such meta-tooling. The bold arrows 
depict meta-tools, and the grey ellipses depict generated code. From a syntax definition, 
a parse component and a pretty-print component are generated that take input terms into 
trees and vice versa. From the same syntax definition a library is generated for each sup- 
ported programming language, which is imported by components that operate on these 
trees. One such component is depicted at the bottom of the picture (more would clutter 
the picture). Several of these components, possibly developed in different programming 
languages can interoperate seamlessly, since the imported exchange code is generated 
from the same syntax definition. 

In this paper we will refine the global architecture of Figure 1 into a comprehen- 
sive architecture for syntax-driven meta-tooling. This architecture embodies the idea 
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Fig. 1. Architecture for meta-tool support for component based language tool develop- 
ment. Bold arrows are meta-tools. Grey ellipses are generated code. 



that grammars can serve as contracts governing all exchange of syntax trees between 
components and that representation and exchange of these trees should be supported by 
a common exchange format. An instantiation of this architecture is available as part of 
the Transformation Tools package XT. 

The paper is structured as follows. In Sections 2, 3, and 4 we will develop sev- 
eral perspectives on the architecture. For each perspective we will make an inventory 
of meta-languages and meta-tools and formulate requirements on these languages and 
tools. We will discuss how we instantiated this architecture: by adopting or developing 
specific languages and tools meeting these requirements. In Section 5 we will combine 
the various perspectives thus developed into a comprehensive architecture. Applications 
of the presented meta-tooling will be described in Section 6. Sections 7, and 8 contain 
a discussion of related work and a summary of our contributions. 



2 Concrete syntax definition and meta-tooling 

One aspect of meta-tooling for component based language tool development concerns 
the generation of code from concrete syntax dehnitions (grammars). Figure 2 shows the 
basic architecture of such tooling. Given a concrete syntax definition, parse and pretty- 
print components are generated by a parser generator and a pretty-printer generator, 
respectively. Furthermore, library code is generated, which is imported by tool com- 
ponents (Figure 2 shows no more than a single component to prevent clutter). These 
components use the generated library code to represent parse trees (i.e. concrete syntax 
trees), read, process, and write them. Thus, the grammar serves as an interface descrip- 
tion for these components, since it describes the form of the trees that are exchanged. 

A key feature of this approach is that meta-tools such as pretty-printer and parser 
generators are assumed to operate on the same input grammar. The reason for this is 
that having multiple grammars for these purposes introduces enormous maintenance 
costs in application areas with large, rapidly changing grammars. A grammar serving 
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Fig. 2. Architecture for concrete syntax meta- tools. The concrete syntax dehnition 
serves as contract between components. Components that import generated library code 
interoperate with each other and with generated parsers and pretty-printers by exchang- 
ing parse trees adhering to the contractual grammar. 



as interface dehnition enables smooth interoperation between parse components, pretty- 
print components and tree processing components. In fact, we want grammars to serve 
as contracts governing all exchange of trees between components, and having several 
contracts specifying the same agreement is a recipe for disagreement. 

Note that our architecture deviates from existing meta-tools in the respect that we 
assume full parse trees can be produced by parsers and consumed by pretty-printers, 
not just abstract syntax trees (ASTs). These parse trees contain not only semantically 
relevant information, as do ASTs, but they additionally contain nodes representing liter- 
als, layout, and comments. The reason for allowing such concrete syntax information in 
trees is that many applications, e.g. software renovation, require preservation of layout 
and comments during tree transformation. 



2.1 Concrete syntax definition 

In order to satisfy our adaptability, scalability and maintainability demands, the concrete 
syntax dehnition formalism must satisfy a number of criteria. The syntax dehnition 
formalism must have powerful support for modularity and reuse. It must be possible to 
extend languages without changing the grammar for the base language. This is essential, 
because each change to a grammar on which tooling is based potentially leads to a 
modihcation avalanche. Also, the grammar language must be purely declarative. If not, 
its reusability for different purposes is compromised. 

In our instantiation of the meta-tool architecture, the central role of concrete syntax 
dehnition language is fulhlled by the Syntax Dehnition Formalism Sdf [11]. Figure 3 
shows an example of an Sdf grammar. This example dehnition contains lexical and 
context-free syntax dehnitions distributed over a number of modules. Note that the ori- 
entation of productions is hipped with respect to BNF notation. 
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definition 




module Main 


module Exp 




imports Exp Let Def 


exports 




exports 


context-free syntax 




sorts Exp 


Identifier 


— >■ Exp {cons(var)} 


lexical syntax 


Identifier “(” {Exp 


“)” —¥ Exp {cons(fcall)} 


[\^\t\n] LAYOUT 


“(” Exp “)” 


Exp {bracket} 


context-free restrictions 


module Let 




LAYOUT? -/- [\^\t\n] 


exports 

context-free syntax 

let Dels in Exp —¥ 


Exp {cons(let)} 




Exp where Dels — > Exp {cons(where)} 





module Def 
exports 
aliases 

{( Identifier “=” Exp ) “ —¥ Dels 

Fig. 3. An example Sdf grammar. 



Sdf offers powerful modularization features. Notably, it allows modules to be mu- 
tually dependent, and it allows alternatives of the same non-terminal to be spread across 
multiple modules. For instance, the syntax of a kernel language and the syntaxes of its 
extensions can he defined in separate modules. Also, mutually dependent non-terminals 
can be defined in separate modules. Renamings and parameterized modules further fa- 
cilitate syntax reuse. 

Sdf is a highly expressive syntax definition formalism. Apart from symbol itera- 
tion constructors, with or without separators, it provides notation for optional symbols, 
sequences of symbols, optional symbols, and more. These notations for building com- 
pound symbols can be arbitrarily nested. Sdf is not limited to a subclass of context-free 
grammars, such as LR or LL grammars. Since the full class of context-free syntaxes, as 
opposed to any of its proper subclasses, is closed under composition (combining two 
context-free grammars will always produce a grammar that is context-free as well), this 
absence of restrictions is essential to obtain true modular syntax definition, and “as-is” 
syntax reuse. 

Sdf offers disambiguation constructs, such as associativity annotations and rela- 
tive production priorities, that are decoupled from constructs for syntax definition it- 
self. As a result, disambiguation and syntax definition are not tangled in grammars. 
This is beneficial for syntax definition reuse. Also, Sdf grammars are purely declara- 
tive, ensuring their reusability for other purposes besides parsing (e.g. code generation, 
pretty-printing). 

Sdf offers the ability to control the shape of parse trees. The alias construct (see 
module Def in Figure 3) allows auxiliary names for complex sorts to be introduced 
without affecting the shape of parse trees or abstract syntax trees. Aliases are resolved 
by a normalization phase during parser generation, and they do not introduce auxiliary 
nodes. 
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2.2 Concrete meta-tooling 

Parsing Sdf is supported by generalized LR parser generation [15]. In contrast to 
plain LR parsing, generalized LR parsing is able to deal with (local) ambiguities and 
thereby removes any restrictions on the context-free grammars. A detailed argument 
that explains how the properties of GLR parsing contribute to meeting the scalability 
and maintainability demands of language-centered application areas can be found in [7]. 
The meta-tooling used for parsing in our architecture consist of a parse table generator, 
and a generic parse component, called sglr , which parses terms using these tables, 
and generates parse trees [16]. 

Parse tree representation In our architecture instantiation, the parse trees produced 
from generated parsers are represented in the Sdf parse tree format, called AsFix [16]. 
AsFix trees contain all information about the parsed term, including layout and com- 
ments. As a consequence, the exact input term can always be reconstructed, and during 
tree processing layout and comments can be preserved. This is essential in the applica- 
tion area of software renovation. 

Full AsFix trees rapidly grow large and become inefficient to represent and ex- 
change. It is therefore of vital importance to have an efficient representation for AsFix 
trees available. Moreover, component based software development requires a uniform 
exchange format to share data (including parse trees) between components. The ATerm 
format is a term representation suitable as exchange format for which an efficient repre- 
sentation exists. Therefore AsFix trees are encoded as ATerms to obtain space efficient 
exchangeable parse trees ([5] reports compression rates of over 90 percent). In Sec- 
tion 3.2 we will discuss tree representation using ATerms in more detail. 

Pretty-printing We use GPP, a generic pretty-printing toolset that has been defined in 
[13]. This set of meta-tools provides the generation of customizable pretty-printers for 
arbitrary languages defined in Sdf. The layout of a language is expressed in terms of 
pretty-print rules which are defined in an ordered sequence of pretty-print tables. The 
ordering of tables allows customization by overruling existing formatting rules. 

The standard distribution of GPP contains a formatter which operates on AsFix parse 
trees and supports comment preservation. An additional formatter which operates on 
ASTs is distributed as part of XT. 

Since GPP is an open system which can be extended and adapted easily, support for 
new output formats (in addition to plain text, LTj^, and HTML which are supported 
by default) and language specific formatters can be incorporated with little effort. 

3 Abstract syntax definition and meta-tooling 

A second aspect of meta-tooling for component based language tool development con- 
cerns the generation of code from abstract syntax definitions. Figure 4 shows the archi- 
tecture of such tooling. Given an abstract syntax definition, library code is generated, 
which is used to represent and manipulate ASTs. The abstract syntax definition lan- 
guage serves as an interface description language for AST components. In other words, 
abstract syntax definitions serve as tree type definitions (analogous to XML’s document 
type definitions). 
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Abstract 

Syntax 

Definition 



T 





Fig. 4. Architecture for abstract syntax meta-tools. The abstract syntax definition, pre- 
scribing tree structure, serves as a contract between tree processing components. 



3.1 Abstract syntax definition 

For the specibcation of abstract syntax we have debned a subset of Sdf, which we call 
AbstractSDF. AbstractSDF was obtained from Sdf simply by omitting all constructs 
specibc to the debnition of concrete syntax. Thus, AbstractSDF allows only productions 
specifying prebx syntax, and it contains no disambiguation constructs or constructs 
for specifying lexical syntax. AbstractSDF inherits the powerful modularity features of 
Sdf, as well as the high expressiveness concerning arbitrarily nested compound sorts. 
Figure 5 shows an example of an AbstractSDF debnition. 

The need to debne separate concrete syntax and abstract syntax debnitions would 
cause a maintenance problem. Therefore, the concrete syntax debnition can be an- 
notated with abstract syntax directives from which an AbstractSDF debnition can be 
generated (see Section 3.3 below). These abstract syntax directives consist of optional 
constructor annotations for context-free productions (the “cons” attributes in Figure 3) 
which specify the names of the corresponding abstract syntax productions. 



3.2 Abstract syntax tree representation 

In order to meet our scalability demands, we will require a tree representation format 
that provides the possibility of efficient storage and exchange. Flowever, we do not want 
a tree format that has an efficient binary instantiation only, since this makes all tooling 
necessarily dependent on routines for binary encoding. Having a human readable in- 
stantiation keeps the system open to the accommodation of components for which such 
routines are not (yet) available. Finally, we want the typing of trees to be optional, 
in order not to preempt integration with typeless, generic components. For instance, a 
generic tree viewer should be able to read the intermediate trees without explicit knowl- 
edge of their types. 

ASTs are therefore represented in the ATerm format, which is a generic format 
for representing annotated trees. In [5] a 2-level API is debned for ATerms. This API 
hides a space efficient binary representation of ATerms (BAF) behind interface func- 
tions for building, traversing and inspecting ATerms. The binary representation format 
is based on maximal subtree sharing. Apart from the binary representation, a plain, 
human-readable representation is available. 
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definition 
module Exp 
exports 
syntax 

“var” ( Identifier ) —¥ Exp 

“fcall” ( Identifier, Exp* ) —¥ Exp 

module Let 
exports 
syntax 

“let” ( Dels, Exp ) — > Exp 

“where” ( Exp, Dels ) —¥ Exp 



module Def 
exports 
aliases 

( Identifier Exp )h — > Dels 

module Main 
Imports Exp Let Def 



Fig. 5. Generated AbstractSDF definition. 



AbstractSDF definitions can be used as type definitions for ATerms by language tool 
components. In particular, the AbstractSDF definition of the parse tree formalism AsFix 
serves as a type definition for parse trees (See Section 2). The AbstractSDF definition 
of Figure 5 defines the type of ASTs representing expressions. Thus, the ATerm format 
provides a generic (type-less) tree format, on which AbstractSDF provides a typed view. 

3.3 Abstract from concrete syntax 

The connection between the abstract syntax meta-tooling and the concrete syntax meta- 
tooling can be provided by three meta-tools, which are depicted in Figure 6. Central 
in this picture is a meta-tool that derives an abstract syntax definition from a concrete 
syntax definition. The two accompanying meta-tools generate tools for converting full 
parse trees into ASTs and vice versa. Evidently, these ASTs should correspond to the 
abstract syntax definition which has been generated from the concrete syntax definition 
to which the parse trees correspond. 

An abstract syntax definition is obtained from a grammar in two steps. Firstly, con- 
crete syntax productions are optionally annotated with prefix constructor names. To 
derive these constructor names automatically, the meta-tool sdf cons has been im- 
plemented. This tool basically collects keywords and non-terminal names from pro- 
ductions and applies some heuristics to synthesize nice names from these. Non-unique 
constructors are made unique by adding primes or qualifying with non- terminal names. 
By manually supplying some seed constructor names, users can steer the operation of 
sdf cons, which is useful for languages which sparsely contain keywords. 

Secondly, the annotated grammar is fed into the meta-tool sdf2asdf, yielding 
an AbstractSDF definition. For instance, the AbstractSDF definition in Figure 5 was 
obtained from the Sdf definition in Figure 3. This transformation basically throws out 
literals, and replaces mixfix productions by prefix productions, using the associated 
constructor name. 

Together with the abstract syntax definition, the converters parsetree2ast and 
ast2parsetree which translate between parse trees and ASTs are generated. Note 
that the first converter removes layout and comment information, while the second in- 
serts empty layout and comments. 
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Fig. 6 . Architecture for meta-tools linking abstract to concrete syntax. The abstract syn- 
tax definition is now generated from the concrete syntax definition. 



Note that the high expressiveness of Sdf and AbstractSDF, and their close corre- 
spondence are key factors for the feasibility of generating abstract from concrete syn- 
tax. Standard, Yacc-like concrete syntax definition languages are not satisfactory in this 
respect. Since their expressiveness is low, and LR restrictions require non-natural lan- 
guage descriptions, generating abstract syntax from these languages would result in 
awkwardly structured ASTs, which burden the component programmers. 

4 Generating library code 

In this section we will discuss the generation of library code (see Figures 2 and 4). 
Our language tool development architecture contains code generators for several lan- 
guages and consequently allows components to be developed in different languages. 
Since ATerms are used as uniform exchange format, components implemented in dif- 
ferent programming languages can be connected to each other. 

4.1 Targeting C 

For the programming language C an efficient ATerm implementation exists as a sep- 
arate library. This implementation consists of an API which hides the efficient binary 
representation of ATerms based on maximal sharing and provides functions to access, 
manipulate, traverse, and exchange ATerms. 

The availability of the ATerm library allows generic language components to be 
implemented in C which can perform low-level operations on arbitrary parse trees as 
well as on abstract syntax trees. 

A more high-level access to parse trees is provided by the code generator asdf 2c 
which, when passed an abstract syntax definition, produces a library of match and build 
functions. These functions allow easy manipulation of parse trees without having to 
know the exact structure of parse trees. These high-level functions are type-preserving 
with respect to the AbstractSDF definition. 
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4.2 Targeting Java 

Also for the Java programming language an implementation of the ATerm API exists 
which allows Java programs to operate on parse trees and abstract syntax trees. As yet, 
there is no code generator for Java available to provide high level access and traversals 
of trees similar to the other supported programming languages. Such a code generator 
has been designed and is being developed. It will represent syntax trees as object trees, 
and tree traversals will be supported by generated libraries of refinable visitors. 

4.3 Targeting Stratego 

Our initial interest was to apply our meta-tooling to program transformation problems, 
such as automatic software renovation. For this reason we selected the transformational 
programming language Stratego [17] as the first target of code generation. Stratego 
offers powerful tree traversal primitives, as well as advanced features such as separation 
of pattern-matching and scope, which allows pattern-matching at arbitrary tree depths. 
Furthermore, Stratego has built-in support for reading and writing ATerms. Stratego 
also offers a notion of pseudo-constructors, called overlays, that can be used to operate 
on full parse trees using a simple AST interface. 

Two meta-tools support the generation of Stratego libraries from syntax descrip- 
tions. The library for AST processing is generated by asdf 2 stratego from an 
AbstractSDF definition. The library for combined parse tree and AST processing is 
generated by sdf 2 stratego from an Sdf grammar. The latter library subsumes the 
former. 

The Stratego code generation allows programming on parse trees as if they were 
ASTs. Underneath such AST-style manipulations, parse trees are processed in which 
hidden layout and literal information is preserved during transformation. This style of 
programming can be mixed freely with programming directly on parse trees. Since 
Stratego has native ATerm support, there is no need for generating library code for 
reading and writing trees. 

4.4 Targeting Haskell 

Work has also been done on targeting Haskell. Code generated in this case is of var- 
ious kinds. Firstly, datatypes are generated to represent parse trees and ASTs. These 
datatypes are quite similar to the signatures generated for Stratego. Secondly, code is 
generated for reading ATerm representations into these Haskell datatypes and writing 
them to ATerms. Finally, full-fledged transformation frameworks consisting of (monadic) 
paramorphisms and corresponding algebras are generated to facilitate purely functional 
transformational programming. The reader is referred to [ 14] for details and for a soft- 
ware renovation case study. 

Note that not only general purpose programming languages of various paradigms can 
be fitted into our architecture, but also more specialized, possibly very high-level lan- 
guages. An attribute grammar system, for instance, would be a convenient tool to pro- 
gram certain tree transformation components. 
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Fig. 7. Complete meta-tooling architecture. The grammar serves as the contract govern- 
ing all tree exchange. 



5 A comprehensive architecture 

Combining the partial architectures of the foregoing subsections leads to the complete 
architecture in Figure 7. This figure can be viewed as a refinement of our first general 
architecture in Figure 1, which does not differentiate between concrete and abstract 
syntax, or between parse trees and ASTs. 

The refined picture shows that all generated code (libraries and components), and 
the abstract syntax definition stem from the same source: the grammar. Thus, this 
grammar serves as the single contract that governs the structure of all trees that are 
exchanged. In other words, all component interfaces are defined in a single location: 
the grammar. (When several languages are involved, there are of course equally many 
grammars.) This single contract approach eliminates many maintenance headaches dur- 
ing component based development. Of course, careful grammar version management is 
needed when maintenance due to language changes is not carried out for all components 
at once. 

5.1 Grammar version management 

Any change to a grammar, no matter how small, potentially breaks all tools that depend 
on it. Thus, sharing grammars between tools or between tool components, which is 
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a crucial feature of our architecture, is potentially at odds with grammar change. To 
pacify grammar change and grammar sharing, grammar management is needed. 

To facilitate grammar version management, we established a Grammar Base, in 
which grammars are stored. Furthermore, we subjected the stored grammars to simple 
schemes of grammar version numbers and grammar maturity levels. 

To allow tool builders to unequivocally identify the grammars they are building their 
tool on, each grammar in the Grammar Base is given a name and a version number. To 
give tool builders an indication of the maturity of the grammars they are using to build 
their tools upon, all grammars in the Grammar Base are labeled with a maturity level. 
We distinguish the following levels: 

volatile The grammar is still under development, 
stable The grammar will only be subject to minor changes due to bug hxing. 
immutable The grammar will never change. 

Normally, a grammar will begin its life cycle at maturity level volatile. To build exten- 
sive tooling on such a grammar is unwise, since grammar changes are to be expected 
that will break this tooling. Once confidence in the correctness of the grammar has 
grown, usually through a combination of testing, bench-marking, and code inspection, 
it becomes eligible for maturity level stable. At this point, only very local changes are 
still allowed on the grammar, usually to fix minor bugs. Tool-builders can safely rely 
on stable grammars without risking that their tools will break due to grammar changes. 
Only a few grammars will make it to level immutable. This happens for instance when 
a grammar is published, and thus becomes a fixed point of reference. If the need for 
changes arises in grammars that are stable or immutable, a new grammar (possibly the 
same grammar with a new version number) will be initiated instead of changing the 
grammar itself. 



5.2 Connecting components 

The connectivity to different programming languages allows components to be devel- 
oped in the programming language of choice. The use of ATerms for the representation 
of data allows easy and efficient exchange of data between different components and 
it enables the composition of new and existing components to form advanced language 
tools. 

Exchange between components and the composition of components is supported in 
several ways. First, components can be combined using standard scripting techniques 
and data can be exchanged by means of files. Secondly, the uniform data representation 
allows for a sequential composition of components in which Unix pipes are used to 
exchange data from one component to another. Finally, the ToolBus [3] architecture 
can be used to connect components and define the communication between them. This 
architecture resembles a hardware communication bus to which individual components 
can be connected. Communication between components only takes place over the bus 
and is formalized in terms of Process Algebra [1]. 
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6 Applications 

Only preliminary experience is available about actually applying the meta-tooling pre- 
sented in the previous sections. We will present a selection of such experiences. 

To start with, the meta-tooling has been applied for its own development, and for 
the development of some other meta-tools that it is bundled with in the Transforma- 
tion Tools package XT. These bootstrap flavored applications include the generation 
of an abstract syntax definition for the parse tree format AsFix from the grammar of 
Sdf. From this abstract syntax definition, a modular Stratego library for transforming 
AsFix trees was generated and used for the implementation of some AsFix normal- 
ization components. Also, the tools sdf 2stratego, sdf cons, asdf 2stratego, 
sdf 2asdf , and many more meta-tools were implemented by parsing, AST processing 
in one or more components, and pretty-printing. 

Apart from Sdf and AbstractSDF, the domain specific languages BOX (for generic 
formatting), and Bench (for generating benchmark reports), have been implemented 
with syntax-driven meta-tooling. In the BOX implementation, a grammar for pretty- 
print tables was built by reusing the Sdf grammar and the BOX grammar. New BOX 
components were implemented in Stratego and connected to existing BOX components 
programmed in other languages. 

The generated transformation frameworks for Haskell are being applied to software 
renovation problems. In [14], a COBOL renovation application is reported. It involves 
parsing according to a COBOL grammar, applying a number of function transform- 
ers to solve a data expansion problem, and unparsing the transformed parse trees. The 
functional transformers have been constructed by refining a transformation framework 
generated from the COBOL grammar. Application to the development of documenta- 
tion generators [10] has commenced. 

7 Related work 

Syntax-driven meta-tools for language tool development are ubiquitous, but rarely do 
they address a combination of features such as those addressed in this paper. We will 
briefly discuss a selection of approaches some of which attain a degree of integration of 
various features. 

- Parser generators such as Yacc [12] and JavaCC are meta- tools that generate parsers 
from syntax definitions. Compared with Sdf and sglr , they offer poor support 
for modular syntax dehnition, their input languages are not sufficiently declarative 
to be reusable for the generation of other components than parsers, and they do not 
generally target more than a single programming language. 

- The language SYN [4] combines notations for specifying parsers, pretty-printers 
and abstract syntax in a single language. However, the underlying parser generator 
is limited to LALR(l), in order to have both parse trees and ASTs, users need to 
construct two grammars, and code the mapping between trees by hand. Moreover, 
the expressiveness of the language is much smaller than the expressiveness of Sdf, 
and the language is not modular. Consequently, SYN and its underlying system can 
not meet our adaptability, scalability and maintainability requirements. 




Grammers as Contracts 



97 



Wile [20] describes derivation of abstract syntax from concrete syntax. Like us he 
uses a syntax description formalism more expressive than Yacc’s BNF notation in 
order to avoid warped ASTs. Additionally, he provides a procedure for transforming 
a Yacc-style grammar into a more “tasteful” grammar. His BNF extension allows 
annotations that steer the mapping with the same effect as Sdf’s aliases. He does 
not discuss automatic name synthesis. 

AsdlGen [19] provides the most comprehensive approach we are aware of to syntax- 
driven support of component-based language tools. It generates library code for 
various programming languages from abstract syntax definitions. It offers ASDL as 
abstract syntax definition formalism, and pickles as space-efficient exchange for- 
mat. It offers no support for dealing with concrete syntax and full parse trees. 
AsdlGen targets more languages than our architecture instantiation does at the mo- 
ment. The choice of target languages, including C and Java, has presumably moti- 
vated some restrictions on the expressiveness of ASDL. ASDL lacks certain mod- 
ularity features, compared to AbstractSDF: no mutually dependent modules, and 
all alternatives for a non-terminal must be grouped together. Furthermore, ASDL is 
much less expressive. It does not allow nesting of complex symbols, it has a very 
limited range of symbol constructors, and it does not provide module renamings or 
parameterized modules. 

Unlike ATerms, the exchange format that comes with ASDL is always typed, thus 
obstructing integration with generic components. In fact, the compression scheme 
of Asdl relies on the typedness of the trees. The rate of compression is significantly 
smaller than for ATerms [5]. Furthermore, pickles have a binary form only. 

The DTD notation of XML [8] is an alternative formalism in which abstract syntax 
can be defined. Tools such as HaXML [18] generate code from DTDs. HaXML 
offers support both for type-based and for generic transformations on XML docu- 
ments, using Haskell as programming language. Other languages are not targeted. 
Concrete syntax support is not integrated. 

XML is originally intended as mark-up language, not to represent abstract syntax. 
As a result, the language contains a number of inappropriate constructs, and some 
awkward irregularities from an abstract syntax point of view. XML also has some 
desirable features, currently not offered by AbstractSDF, such as annotations, and 
inclusion of DTDs (abstract syntax definitions) in documents (abstract terms). 
Many elements of our instantiation of the architecture for syntax-driven component- 
based language tool development were originally developed in the context of the 
Asfh-Sdf Meta-Environment [2,11,9]. This is an integrated language development 
environment which offers Sdf as syntax definition formalism and the term rewrit- 
ing language ASF as programming language. Programming takes place directly on 
concrete syntax, thus hiding parse trees from the programmers view. Programming, 
debugging, parsing, rewriting and pretty-printing functionality are all offered via a 
single interactive user interface. Meta-tooling has been developed to generate ASF- 
modules for term traversal from Sdf definitions [6]. 

The Asfh-Sdf Meta-Environment offers a single programming language (Asf), 
programming on abstract syntax is not supported. Support for component-based 
development is (currently) limited to gluing compiled ASF programs that read and 
write flat terms. 
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To provide support for component-based tool development, we have adopted the 
Sdf, AsFix, and ATerm formats from the Asf-hSdf Meta-Environment as well 
as the parse table generator for Sdf, the parser sglr , and the ATerm library. To 
these we have added the meta-tooling required to complete the instantiation of the 
architecture of Figure 7. In future, some of these meta-tools might be integrated 
into the Meta-Environment. 

8 Contributions 

We have presented a comprehensive architecture for syntax-driven meta-tooling that 
supports component based language tool development. This architecture embodies the 
vision that grammars can serve as contracts between components under the condition 
that the syntax dehnition formalism is sufficiently expressive and the meta-tools sup- 
porting this formalism are sufficiently powerful. We have presented our instantiation of 
such an architecture based on the syntax formalism Sdf. Sdf and the tools supporting 
it have agreeable properties with respect to modularity, expressiveness, and efficiency, 
which allow them to meet scalability and maintainability demands of application areas 
such as software renovation and domain- specihc language implementation. We have 
shown how abstract syntax definitions can be obtained from grammars. We discussed 
the meta-tooling which generates library code for a variety of programming languages 
from concrete and abstract syntax definitions. Components that are constructed with 
these libraries can interoperate by exchanging ATerms that represent trees. 
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Abstract. Software reuse and especially the paradigm of software components 
are promising approaches to increase the efficiency of software development. 
One of the basic problems of software reuse, however, is the tradeoff between 
the abstraction from project-specific aspects and the actual contribution of a re- 
usable artifact. Stringent resource constraints further complicate the application 
of these approaches in domains, where efficient and therefore specialized solu- 
tions are required, e.g. in the domain of embedded systems. Generic compo- 
nents - designed to be adaptable to new application scenarios - allow to over- 
come these limitations, esp. if they automate the essential modifications. This 
paper presents a concept of generic components that has been developed to fa- 
cilitate the construction of highly specialized embedded operating systems. Be- 
sides the illustration of the underlying concept, the paper discusses the external 
representation of generic components and the internal realization of the required 
variability and reflects some of our experiences in constructing generic compo- 
nents. 



1 Introduction 

The highly competitive and dynamic field of software development implies the need 
to continuously increase the efficiency of the construction and evolution of software 
products. For a long time reuse of software artifacts in general and of standard com- 
ponents in particular have been promising approaches [ 16 , 18 ] to achieve this goal 
through the consequent and efficient exploitation of the commonalities among pro- 
duced software artifacts. However, there are inherent problems with both techniques. 
Firstly, software artifacts have to be made sufficiently abstract to be reusable, i.e. to 
be applicable in different deployment situations, while at the same time need to be 
sufficiently concrete to bear a reuse benefit at all. With component-based software 
development this gets even worse, since the components are typically intended to be 
reused in a black box fashion and thus often do not support their customization at 
compile-time in regard to individual demands. Secondly, the evolution of prefabri- 
cated software artifacts - e.g. caused by changing requirements - in most cases results 
in a rise of complexity that further degrades the performance and thus often prohibits 
their reuse. This holds especially if tight resource constraints have to be met, e.g. in 
the domain of embedded systems. 
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To deal with the required variability, generative programming teehniques, e.g. as 
presented in [7], are sound approaehes, sinee they also pursue reuse on the level of the 
produetion proeess instead solely on the produet level. The variability among the 
artifaets thus ean be more effeetively managed and even be formulated in the eontext 
of the problem spaee. But there are problems with pure generative approaehes, too. 
Domain-speeifie default assumptions have to be applied, otherwise the eomplexity of 
the problem speeifieation and the underlying generator would be of the same magni- 
tude as the implementation eomplexity of the whole problem itself Again, the trade- 
off between default solutions and the need for individual optimizations is likely to 
appear. As a eonsequenee manual speeializations have to be eonsidered within the 
reuse aetivities as well, if the effieieney of the results is essential. The ehallenge is to 
eombine different reuse strategies in a way that exploits the benefits of eaeh approaeh. 
Code generation ean be used to supplement the reuse of software eomponents by 
automating their otherwise time-eonsuming and error-prone eustomization. Pre- 
planned manual optimizations within the reusable artifaets ean be supported by pro- 
viding adequate implementation eonstraints and knowledge, ete. 

We have developed an approaeh based on the eomponent and the generation para- 
digm in the domain of embedded operating systems to address the aforementioned 
problems. It has been eoined by the stringent resouree eonstraints, sueh as searee 
memory and low eost CPUs, in eombination with varying nonftmetional require- 
ments, like robustness and timeliness. The resulting effieieney eonstraints - often the 
most erueial eriterion for the sueeess or failure of a software projeet within this do- 
main - make eonventional reuse approaehes espeeially problematie in this eontext, 
sinee they neeessitate even unique tailor-made solutions that are very hard to reuse 
effieiently in following projeets. To address this problem, the eoneept of generic 
components has been developed. Generie eomponents provide means to ease their 
tailoring aeeording to individual requirements; they eombine the teehnique of reusing 
prefabrieated software artifaets with the goal of produeing speeialized software. Most 
of the variability provided by them ean be automated by generators aeeomplishing 
even fine-grained optimizations on the level of souree eode, but also unique eustomi- 
zations are supported. Obviously, the eombination of the eomponent and the genera- 
tion paradigm only beeomes possible through a weakening of the eonventional eom- 
ponent notion. With generie eomponents also grey-box reuse has to be eonsidered. 

Although this integrative eoneept has been developed for the domain of embedded 
operating systems, it should also be applieable in other domains - provided that a 
known and manageable set of eomponent variants is likely to eover the majority of 
possible requirements. This paper presents our eoneept of generie eomponents and 
foeuses on their external representation and some general faets eoneeming the internal 
realization of the provided variability. The rather proeess-related questions of map- 
ping requirements to appropriate eomponent seleetions and eonfigurations in our 
approaeh is eovered by eompanion papers [3,4]. 

The remainder of this paper is struetured as follows: Seetion 2 introduees the eon- 
eept of generie eomponents in-depth. Seetion 3 eovers the external representation of 
generie eomponents. In Seetion 4 several approaehes to implement generie eompo- 
nents are diseussed. Finally, in Seetion 5, we draw some eonelusions. 
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2 The Concept of Generic Components 

Conventional component-based software development not considering customizations 
at compile-time reaches its limits where tailor-made solutions are required to meet 
stringent nonfunctional requirements. To address this, the concept of generic compo- 
nents [2] has been developed, an approach for delivering software solutions that - 
despite of relying on the reuse of components - are customized to the settings of a 
specific application domain. Generic components combine the techniques of reusing 
prefabricated software artifacts with the goal of producing specialized software. 

A generic component can be defined as a software module allowing to choose its 
properties to a certain degree without necessarily having to write or change code 
manually. The choice of properties may concern both functional and nonfunctional 
aspects, e.g. the choice of algorithms or data structures and the adaptation of the com- 
ponent's interface. To support this 
kind of customization, so-called 
generic parameters together with 
their respective range of supported 
values are identified at the compo- 
nent's customization interface. These 
are the externally visible setscrews to 
control the instantiation of the ge- 
neric component into a conventional 
one (cf Fig. 1). Within the generic 
components so-called variation 
points - placeholders for the variable 
parts of code - can be identified that 
are replaced with concrete code 
fragments produced by generators in accordance to the actual values chosen for the 
generic parameters during the instantiation. Typically only few parameters have an 
impact on a variation point; they form the input to the specific generator that is asso- 
ciated with this variation point. To subsume, customizing a generic component hence 
means to choose actual values for the supplied generic parameters, and let appropriate 
generators accomplish the necessary modifications. Besides a pure reduction of cus- 
tomization effort, customizations on a higher level of abstraction can be supported 
through the application of generative techniques, e.g. template meta programming, or 
sophisticated code compilers. The application developer, for instance, applying ge- 
neric components to construct embedded control systems does not need to know in 
detail what changes within the components are required to provide thread preemption 
or a delay ( ) functionality. He only has to decide if the functionality is needed and 
has to be informed about the additional resource requirements. 

From the conceptual point of view a generic component is a kind of abstract repre- 
sentative of a set of closely related software components with slightly different prop- 
erties. Since it is only sensible to integrate such components in a generic component 
that share a considerable amount of common code, generic components - according to 
[17] - can be considered as a component family. The different family members can be 
distinguished by the tuple of actual values, chosen from the subset of the Cartesian 
Product over the generic parameters that contains the consistent parameter selections. 




Fig. 1. Conceptual view on generic components. 
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As with program families, a thorough analysis of the presumably needed properties, 
and the resulting commonalities and variabilities should take place before generic 
components are implemented. Domain Analysis methods [8, 18] or the Multi- 
Paradigm Design [6] can be used to this end. 



2.1 Generic Parameters 

Generic parameters - the pendants of run-time configuration parameters at compile- 
time - allow to tailor the component’s properties by simply choosing one of the sup- 
ported parameter values. Trivial properties like the value of a constant can be instanti- 
ated as well as code fragments realizing special data structures or algorithms. Specific 
adaptations concerning the runtime environment of the component, e.g. the type of 
used hardware, can thus be handled within the component, instead of providing exter- 
nal wrappers causing additional costs. Especially in the case of directly interacting 
components, using function calls to communicate with each other, adaptations of the 
calling interface are very useful. Furthermore, the interfaces of generic components 
can be the target of the customization. Superfluous complexity [5] can be reduced in 
this way by providing only the needed functionality. Finally, a tailoring of nonfunc- 
tional properties of the component can be supported, e.g. an optimization concerning 
the processing time or memory consumption at run-time. This tailoring is usually 
driven by the wish to construct software components that meet the essential given 
requirements and moreover support as many as possible of the desired properties, e.g. 
little resource demands, reduction of the interface complexity, etc. 

Three major approaches to tailor code that is used to replace a variation point can 
be identified (cf Fig. 2). Variation points are filled by selection if they are simply 
replaced with discrete entities that are selected from a finite set of alternatives. Enti- 
ties in this sense are pre-built parts of a 
component implementation, e.g. data 
structures, algorithms, or other code 
fragments. In the case of generation, 
the variation points are placeholders for 
code fragments produced by associated 
generators depending on the provided 
actual values of the generic parameters. 

The range of generator functionality 
can reach from the simple insertion of 
values for constants over the expansion 
of small macros to sophisticated code 
generation, where abstract descriptions 
in form of the actual values are trans- 
lated into concrete artifacts. With this type any generative technique can be utilized in 
generic components. Where the aforementioned tailoring types do not provide suffi- 
cient flexibility, variation points can be replaced by parameter substitution, i.e. a 
variation point is substituted with the actual value of the respective parameters. In this 
way, user-defined code can be inserted into the component. This type of tailoring 
offers the highest degree of flexibility to the user and is typically used to handle code 




Fig. 2. Different types to tailor code that 
replaces a variation point. 
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fragments that are unique in each component instantiation, e.g. individual optimiza- 
tions. Besides the pure replacement of the parameter an important and nontrivial task 
during the instantiation of a generic component can be the monitoring of the insertion. 
In this case, the generator checks the provided code against some constraints or prede- 
fined limits formulated by the implementer of the component. If a generic parameter 
only controls customizations of the same type, esp. in the case of substitution, it is 
sensible to speak of a selection, generation or substitution parameter. The mentioned 
tailoring types represent different kinds of reuse. While a selection reuses concrete 
artifacts, generation reuses the production methods, and substitution supports the 
reuse of knowledge and constraints about prospective solutions. 

In principle, it is reasonable to perform the customization hierarchically, i.e., new 
generic parameters can be introduced with the code fragments that replace variation 
points. However, a recursive introduc- 
tion of new parameters while choosing 
parameter settings adds a considerable 
amount of complexity to the configura- 
tion process. Therefore, it is sensible to 
restrict the introduction of new generic 
parameters to the tailoring by selection 
as illustrated in Fig. 3. A typical sce- 
nario where hierarchical parameter 
introductions are used is the following: 
a fixed number of implementations exist 
for a variation point, e.g. different 
hardware drivers , but the opportunity to 
add a new implementation has to be 
given to the user, too. This can be realized with a selection between the existing code 
fragments and a frame that provides a substitution parameter. If the user decides to 
provide new code, he selects the frame and afterwards supplies the new code by sub- 
stituting the new parameter. 




Fig. 3. Hierarchical tailoring: introduction of a 
new parameter though selection. 



2.2 Objectives of the Approach 

The concept of generic components has been developed to make the component para- 
digm applicable even in domains where the black-box reuse of components typically 
will not work, i.e. that tailor-made solutions are required for some reasons. Through 
the identification of variation points and of generic parameters the customization at 
compile time is supported allowing the tailoring of components in respect to stringent 
requirements. Doing so, the generic components help to overcome the inherent ten- 
sion between reusability and usefulness of conventional components. If feasible, the 
tailoring complexity can be captured in pre-built fragments or sophisticated genera- 
tors providing implementations for the variation points’ instantiation. Otherwise sub- 
stitution parameters can be used to support the creation of new specializations through 
the clear identification of the relevant variation points and the specification of imple- 
mentation constraints and knowledge. Some kind of abstract components can be real- 
ized in this way that are very useful in the case of highly individual and therefore not 
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reusable aspects, or yet unknown requirements and solutions. The clear identification 
and the factorization of the common and variable parts is advantageous for the follow- 
ing reasons: Firstly, the evolution of the generic components gets easier, e.g. as com- 
mon code is not replicated in several component variants or as new variants created 
by the user can be easily integrated into the respective generators. Secondly, the num- 
ber of related components in reuse pools and therefore the complexity of their re- 
trieval can be reduced in case of orthogonal variabilities. If a thread manager compo- 
nent, for instance, offers generic parameters for the number of supported threads and 
the kind of the used stacks, then a lot of different thread manager variants would have 
to be in the reuse pool in order to meet stringent requirements. In the case of a generic 
component only one component with corresponding generators is needed. With the 
ability to flexibly adapt the implementation of the reused components, conventional 
approaches to improve component reusability, which typically interfere with the over- 
all system performance, can be avoided, e.g. the strict uncoupling of components, and 
large inheritance hierarchies. To subsume, the concept of generic components ap- 
proximates the advantages of manual customization and optimization, while on the 
other hand sidestepping the obstacles of time-consuming and error-prone coding by 
reusing prefabricated and tested implementations. 



3 The User’s Perspective on Generic Components 

The reusability of software components strongly depends on their external representa- 
tion. Comprehensive descriptions of the provided functionality, the interfaces, the 
dependencies on other artifacts and their architectural embedding, etc are essential, 
especially as the components are typically reused in a black-box fashion. This also 
holds for generic components. Due to the variability provided by the generic compo- 
nents, however, further information to represent the generic component in a compre- 
hensive manner is required. From an external point of view, generic components 
mainly differ from conventional ones in the provision of an additional customization 
interface and in the fact, that some of the components' properties are kept variable. 
Furthermore, some mechanism is supplied that supports the automatic tailoring of the 
component according to the actual values chosen for the generic parameters. In the 
following, however, we will concentrate on the customization interface and elaborate 
the required information that has to be presented to the user of the generic component 
to support its consistent configuration and application. 



3.1 Customization Interface 

A generic component in our sense is a kind of abstract representative of a family of 
software components. It can be put into concrete shapes with the assignment of actual 
values to the provided generic parameters. Therefore, at least the generic parameters 
must be clearly identified to give the user an idea what can be tailored at all - but that 
will not suffice. To scope the variability of the parameters, the range of supported 
values for each of them has to be specified as well as the interdependencies among 
them, typically restricting the number of consistent parameter selections. As the actual 
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values of substitution parameters are directly inserted into the generic component, it is 
impossible to enumerate all supported values for this kind of parameter - otherwise 
the substitution parameter should be replaced by a selection parameter. However, at 
least the language and an abstract description of the expected code fragment can be 
stated to facilitate the consistent parameter assignment in that case. 

Since the users of generic components are naturally more interested in the results 
of their tailoring than the tailoring itself, further information regarding the variable 
properties should be given too. In our approach, this information is structured in so- 
called feature types and features. A feature type comprises clearly distinguishable 
features and represents a variable property of the component, e.g. scheduling strategy, 
memory management strategy, or synchronization mechanism. Feature types thus 
span the space of components that can be instantiated from a generic component. This 
information, together with the description of the common features, is typically used to 
determine in a very early stage, whether a generic component is applicable for the 
specific problem or not. 

Obviously the feature types of a generic component are somehow correlated with 
its generic parameters. If this correlation is explicitly represented in the customization 
interface, it can be used to determine the concrete features of the tailored component 
before the final instantiation step. This feedback can be used for composability tests 
and for further adjustment of the component, and therefore facilitates the selection of 
an appropriate 
actual parameter 
set. For each fea- 
ture type, a func- 
tional dependency 
on a subset of the 
generic parameters 
can be identified. In 
our approach, this 
set of feature- 
dependency func- 
tions is used to 
represent the corre- 
lation between a feature type and its parameters. One special case is worth to be men- 
tioned: feature-dependency functions that are invertible can be used favorably to 
determine parameters based upon the desired features. This significantly facilitates the 
customization of a component. Fig. 4 subsumes the aforementioned points into a 
formal representation of the customization interface. As the comprehensible descrip- 
tion of the listed aspects is non-frivial and expensive, it may be reasonable to provide 
less information at the cost of user support, e.g. the feature-dependency functions 
could be omitted. 



Representation of the customization interface Cl: 



Cl = df (P, (Pi), VS, F, (Fi), (FDi)) 

With: I 

P =df {Pi, ■■■ ,Pn}, the set of generic parameters 

Pi =df {Pi, ■■■ , Pm(i)}, supported values for each parameter 

PP =df Pi X ... X Pn, the space of parameter selections 

VS =df {pp e PP I PP is a valid selection}, the set of valid selections 

F =df {Fi, ... , Fk}, set of feature types 

Fi =df {fi, ■■■ , fm(i)}, variety of features for the feature type Fi 

PF = df Fi X . . . X Fk, the space of varying features 

FDi =df VS ^ Fi, feature-dependency function 

Fig. 4. Representation of the customization interface. 



3.2 Parameter Dependencies 

After the discussion of the information that has to be presented at the configuration 
interface, we take a closer look at the configuration step itself In practice, the generic 
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parameters often are not independent of each other. Typically, it is possible to identify 
hierarchical dependencies between them; i.e. the presence of some generic parameter 
depends on the choice of actual values for others. For instance, a generic parameter 
that is only supplied in the context of a distinct data structure does not have to be 
considered if the data structure is not used in an actual configuration. Although this 
information is already represented by the set of valid selections, the customization 
complexity for the user can be further reduced if these parameter dependencies are 
explicitly considered during the configuration step. Instead of a flat configuration, 
where all parameters are visible throughout the customization, although a part of them 
is not really needed, a hierarchical configuration should be preferred. Here only those 
parameters that bear a meaning for the current level of choices are presented. This 
approach typi- 
cally results in 
less parameters 
and a more 
intuitive way of 
configuration, 
but requires 
more support in 
the customiza- 
tion interface 
and the instan- 
tiation of the 

component. Eventually some backtracking has to be done in the case that a lower 
level of configuration reveals the insufficient selection of a parameter on a higher 
configuration level. In Fig. 5 an example to illustrate the different configuration ap- 
proaches is given. The arrow between A and B denotes that B only has to be consid- 
ered in the case A is set to 2. Obviously, the hierarchical approach facilitates the con- 
figuration, e.g. no values have to be selected for parameters B and E unless the value 
2 is chosen for A. The example additionally points out the two major reasons for 
hierarchical dependencies. They are either caused by an implicit requires-dependency 
as shown between A and B, or hierarchical tailoring (cf Section 2) as with A and C. 



Flat configration Hierarchical 
configration 

A=?, B=?, A=? 

C=?, D=?, / I \ 

E=? A=1 A=3 A=2 

|E]^ cioiTBy 

a rii Si s 

Fig. 5. Parameter dependencies and the different configuration approaches. 



Parameter dependencies 
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3.3 Tool Support 

Several approaches to represent the customization interface and to support the selec- 
tion of actual values in a user-friendly way are reasonable. However, it becomes clear 
that tool support, esp. for the feature-dependency functions and hierarchical configu- 
ration, is indispensable. Therefore, we apply tools [3] based on the technique of ex- 
tended design-spaces, a multi-dimensional classification scheme that allows to ex- 
press such correlations between features and parameters as well as hierarchies of 
parameters in a very natural and concise way [1, 14, 15]. The required information is 
kept in so-called component spaces that are used by our tools to control the customi- 
zation process. Besides the customization of generic components, a mechanism has to 
be supplied that supports the automatic tailoring of the component in correspondence 
with the chosen configuration. These two mechanisms can favorably be integrated 
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into a single tool, thus automating the customization and the instantiation of generic 
components. Our D-Space-1 tool [3], for instance, facilitates the application of the 
customization interface of our generic components through a graphical interface and 
controls the tailoring of the generic components by calling the appropriate generators, 
which accomplish the specified modifications. 



4 Internals of Generic Components 

From the implementers’ point of view, the descriptive elements of generic compo- 
nents - especially their code - significantly differ from those of conventional ones. 
Generic components comprise a set of variable implementations and generic parame- 
ters to determine them. In addition to the conventional description elements, such as 
code, header files, and resources, the required mechanisms to automate the tailoring 
must be supplied in some way by the implementer of the generic component. A 
wealth of well-known possibilities addressing different facets like code organization 
and modification, customization of structure and behavior, extensibility of existing 
implementations, or code generation are available for this kind of meta programming, 
e.g. appropriate language constructs, pre-processors, external generators. All of these 
viable implementation approaches have to somehow integrate the common and the 
variable description parts that form a generic component. Before discussing the dif- 
ferent tailoring techniques themselves, we want to look at the preconditions needed to 
implement the tailoring facilities in the generic components. 



4.1 Integration of Commonalities and Variabilities 

Different techniques can be applied to tailor the description fragments that replace the 
variation points in the concrete component implementation. All of these approaches 
have at least two concepts in common. Firstly, the variable description parts have to 
be associated with the common ones in some way. Secondly, links between the inter- 
nally used generic parameters and their externally visible pendants at the customiza- 
tion interface have to be established to propagate the external settings. Besides the 
mere description of the variability, the following aspects thus have to be addressed in 
some manner: the identification of the variation points, the association of tailoring 
transformations, and the representation of the generic parameters. Although the clear 
identification of the variation points might seem to be dispensable in some cases, esp. 
if the same language is used to describe the common and variable parts of a generic 
component, it is extremely helpful if the variability of the generic component has to 
be revised or new variants have to be added. With this little overhead, tool support for 
the localization of variable implementations throughout the generic component can 
easily be provided. The variation points can be identified by a special markup that 
obviously must not be used within the description of the common parts. For this rea- 
son, the number of different markups should be kept as small as possible. The coher- 
ent use of unique markups further eases the construction of tools for the identification 
of the variation points within the generic component in that they do not have to be 
aware of the actual techniques applied to describe the variability. 
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Concerning the association of the tailoring transformations with the variation points, 
two major methods are conceivable: inline and link. The tailoring is specified inline, 
if it is described directly in the variation point. Since this results in a mixture of dif- 
ferent kind of description types in most of the cases, doing so is only advisable if the 
transformation is not too complex and cannot be applied elsewhere. More flexibility is 
provided through an external tailoring description or a stand-alone tool that is linked 
with the variation points. This link can be realized with an entry in the variation point 
that is used to call the external generator, or an external identification of the variation 
point that can be handled with a unique variation point identifier. In the latter case, 
several variation points can be substituted with one transformation. As generic pa- 
rameters typically affect multiple variation points, the required elementary transfor- 
mations sometimes can be combined into one comprehensive transformation. 

As stated above, the generic parameters that control the instantiation of a variation 
point have to be represented in a way that allows the propagation of the respective 
actual values provided at the customization interface. If a transformation is specified 
inline no further representation of the generic parameters is required, since they can 
be identified and replaced within the transformation descriptions if a uniform naming 
scheme is used. If a transformation is associated with a variation point via a link, the 
required parameters have to be specified along with the link somehow, e.g. as call 
parameters of the external tool. 



4.2 Tailoring Techniques 

After the discussion of methods to integrate the common and variant parts of generic 
components, we will now focus on the tailoring techniques applicable to realize the 
variability. As the concept of generic components is primarily aimed at compile-time 
tailoring, almost any text transforming or text generating technique can be deployed. 
Based on our experience, the necessary tailoring can be favorably realized through a 
combination of the following techniques: 

Separate files - separately implemented variant parts of components are selected 
by choosing the corresponding source file for compilation and linking. This is the 
most simple approach; neither modifications of the descriptions nor special actions for 
instantiation are necessary. The selection of the files can be accomplished through 
external file management tools like RCS or CVS. 

Pre-processors - scan the descriptions for particular tokens - e.g. programming 
language constructs or specific pre-processor statements -, parse their semantic con- 
text, and perform associated pre-defined actions. In their full extent, pre-processors 
are problem-specific compilers similar to those for simple programming languages 
and can be built using scanner and parser generators like lex/yacc. With special pre- 
precessors other programming paradigms such as Aspect-Oriented Programming [13] 
can be applied within generic components, e.g. to realize efficient implementations of 
variable nonfunctional aspects. However, a great deal of variability can be realized 
with the pre-processor functionality provided by the programming languages or 
scripting tools like sed or perl in a very straightforward way. 

Generic mechanisms of programming languages - most programming languages 
and the corresponding compilers somehow support generic programming, e.g. 
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through object-orientation, templates, or compile time parameters and optimizations. 
Wherever it does not contradict efficiency and comprehensibility, the deployment of 
such mechanisms is a natural and well-known way to realize the required variability. 
Examples are the approaches of template meta programming [20] [19] and generative 
programming [7] that both extensively utilize the template mechanism offered by 
C++ to build variable implementations. One drawback of these mechanisms is that the 
implementation of the common and the variable parts are mixed up, thus complicating 
the location of the variable implementations in most cases. 

Code generators - the most powerful, but also most complex way to perform the 
tailoring. Higher level descriptions of the desired functionality are passed through the 
generic parameters to external generators producing tailored description parts, which 
afterwards replace the correspondig variation points. The use of generators is espe- 
cially tempting wherever the modification of pre-built components does not appear to 
be sufficient to cope with heavily varying and unforeseen requirements. One problem 
with generators is that they must apply domain-specific defaults for those aspects that 
are not explicitly specified by their input, mostly resulting in code that in less efficient 
as manually optimized code. 



4.3 A Glance at Implementations 



TCBptr old; 

old = running [n] ; 


// 


GP2 


Mil TP: CHANGE 


0) 


old->setLastCPU (n) ; 


// 


GP2 


M13 TP: REMOVE 


"D 

O 


// GPS M7 TP: INSERT 








0) 


ProcContext . All . Save (old- >TCB . Register) ; 


u 

3 


return old; 








o 

(/) 



} 

s/\ (Dispatcher\ . [he] [he] -\)/\l Single Proeessor -/ 



In the following, two approaches to implement the tailoring of a generic component 
are exemplarily presented. In the first example given in Fig. 6, the UNIX stream edi- 
tor sed [9] is employed to tailor the get function of a generic dispatcher that supports 
multi -processor environments. To allow for optimizing the source code for single- 
processor envi- TCBptr Dispatcher :: get (slze_t n){// GP2 M10 TP : CHANGE 
ronments, refer- 
ences to the pa- 
rameter n have 
been marked by 
comments of the 
form GPx My. An 
appropriate sed 
script can search 
for the respective 
lines of source 
code and apply 
whatever changes 
are necessary. 

According to the 
sed command 
syntax, line 3 for 
instance searches 
for lines with the 
mark GP2M10, 
then takes the first 
occurrence of the 



/GP2_M10/ 

/GP2_M11/ 

s/\/\/GP2_ 



s/\ (size_t n\) // 
s/\[n\]// 

M13// 



u 

(/) 

■D 

0 ) 

U) 




Fig. 6. A small excerpt from a generic dispatcher component together 
with a sed script allowing to tailor the source code. 
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string “size t n“, and replaces it by an empty string. Variation points and the corre- 
sponding generic parameters are identified with the GPX MXX markup. The tailor- 
ing description is specified externally in the sed script and is therefore associated via 
links - the markup in this case - with the variation points. It may be questioned, how 
the actual values of the generic parameters are considered in this approach. Appar- 
ently, these values must be propagated into the script. For this, a generic sed script is 
provided along with the generic component (cf Fig. 6). In this script, placeholders for 
the generic parameters are identified. After the parameter assignment the generic 
parameters in the generic sed script are replaced by the actual values. The result of 
this replacement is a sed script that is finally used to tailor the component. 

In the next example, 
a quite different ap- 
proach is presented. 

Figure 7 shows an 
excerpt of a generic 
scheduler component. 

Java is used as a meta- 
programming language 
to implement the cus- 
tomization. The tailor- 
ing is described inline 
in the source code. All 
lines that start with a //. 
mark contain instruc- 
tions of the Java meta- 
program. In lines 1-2, 
for instance, the ge- 
neric parameters are 
declared, and lines 5-7 
show an if-block. The 
meta-programming 
results in a source code 
description denoted as 
pseudo code that can be transformed through a simple generator into an executable 
Java class. Advantageous in this approach is that, although a kind of pre-processing is 
used to implement the tailoring, the complete expressiveness of a full-fledged pro- 
gramming language is available, and that finally an executable class is generated, 
which encompasses the generic component and automates the required tailoring. Any 
external generator can be called from within the pseudo code, and interactive support 
for substitution parameters becomes feasible. Conceptually, this approach is closely 
related with the work presented in [12] in the way that a general purpose program- 
ming language is used to automate variability within components. The both ap- 
proaches differ in the scope of the meta programming, since we use this technique 
only to implement the generic components not to compose them. Our composition 
approach can be found in [3]. 



//. generic parameters: 

//. String schedStrategy ; ^ 

[•••] ■§ 

SCHEDULER C :: -SCHEDULER CO { “ 

//. if ( schedStrategy. equals ( "Prio" ) ) { ~o 

delete [] TABLE; ® 

//. } “• 

} 

[. . .] 

void SCHEDULER_C: :ReadyToRun(THREAD_P pclThread) { 
int _iPriority = pclThread - >GetPriority () ; 

//. if { schedStrategy . equals { "Prio" ) ) { 

//. includeFile { "ReadyToRun_prio . cpp" ) ; 

//. } else if { schedStrategy . equals { "EDFPrio" )) { 
//. iEntries++; } 




Fig. 7. Generic scheduler component that is transformed into an 
executable generic component automating its own configuration. 
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5 Conclusions 

The presented concept of generic components aims at improving the reusability of 
prefabricated software components by explicitly planning for specific types of fore- 
seeable adaptations, and by providing mechanisms and tools to perform such adapta- 
tions automatically. Generic components are implemented to be easily modified 
within prescribed bounds and to expose generic parameters, which provide the exter- 
nal handles for choosing and fine-tuning component properties. Tools like configura- 
tion management systems, pre-processors, or full-fledged code generators then allow 
to flexibly customize components for specific application environments with mini- 
mized manual intervention. Although the individual techniques used to implement 
and customize generic components are all well known, their explicit combination 
within the concept of generic components presents a new and promising way to pur- 
sue component reuse, even in the domain of embedded systems. With the ability to 
flexibly adapt the implementation of reused components, conventional approaches to 
improve component reusability, which typically interfere with the overall system 
performance, can be avoided. The concept of generic components approximates the 
advantages of manual customization and optimization, while on the other hand side- 
stepping the obstacles of time-consuming and error-prone coding by reusing prefabri- 
cated and tested implementations. 

We have applied the presented techniques and the notion of generic components to 
build a number of reusable building blocks for embedded operating systems. As soft- 
ware reuse is especially beneficial - not to say: only works - within one type of archi- 
tecture [10], these building blocks are designed for specific reference architectures, 
respectively. For a small operating system kernel for embedded systems, for instance, 
a total of 12 generic components provide the flexibility to generate more than a hun- 
dred different kernel variants covering a wide spectrum of possible requirements. 
Thanks to the fine-grained modifications that are applied to the generic components 
during instantiation, the resulting system properties - especially concerning critical 
nonfunctional aspects such as the timing behavior and the memory consumption - are 
comparable to manually tailored implementations. On the downside, however, the 
implementation effort for generic components is significantly higher than for conven- 
tional components. According to our experience, making component implementations 
generic approximately accounts for additional 50 to 100% of the development effort 
as compared to conventional implementations. And for most cases, this is just half the 
way: in order to further support the selection and configuration of components by 
tools as presented in chapter 3 or in [3], comprehensive component descriptions are 
required. Such descriptions roughly make up another 100% of the initial effort for 
conventional implementations, ultimately summing up to tripled development costs 
for generic components - an investment, after all, that is typically compensated by the 
increased probability of reuse. To further reduce this overhead, we are currently ex- 
tending our research to the questions of how to specifically support the development 
of generic components. The possibilities range from ergonomic editors smartly han- 
dling the variant parts of the source code, to automated detection of component inter- 
dependencies and automatic deduction of properties of composed artifacts. 
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Abstract Component-orientation is an emerging paradigm that pro- 
mises components that are usable as prefabricated black-boxes. But com- 
ponents have the problem that they should be changeable and flexibly 
adaptable to a huge number of different application contexts and to 
changing requirements. We will argue, that sole parameterization - as 
the key variation technique of components - is not suitable to cope with 
all required change scenarios. A proper integration with multiple other 
paradigms, such as object-orientation, the usage of a scripting language 
as a flexible component glue, and the exploitation of high-level inter- 
ception techniques can make components be easier (ex)-changeable and 
adaptable. These techniques can be applied without interfering with the 
component’s internals. 

1 Introduction 

The task of a software engineering project is to map a model of the real world 
(existing or invented) onto a computational system. The complexity and diversity 
of concrete real world systems can be overwhelming. This is no complexity in the 
algorithmic sense, but an complexity of an overwhelming amount of details and 
of particularities in the universe of discourse. By developing a model we reduce 
this complexity by finding and extracting commonalities. The key instruments 
of modeling are abstraction and partitioning. Analyses of commonalities let us 
understand the common elements of a targeted system. The aim of any analysis 
of commonalities is to group related members of a family, regardless whether the 
members are components, objects, modules, functions, etc. 

Orthogonal to the task of modeling commonalities (where details are re- 
moved) is the task of engineering variability. It makes absolutely no sense to 
create abstractions to understand a family as a whole, if we do not introduce 
proper means for variation in the family members [3]. Finding commonalities 
in software eases understanding and reduces the need for changes, while finding 
proper variabilities enables us to use the software at all, because we have to 
re-adapt the found abstractions to the concreteness of the modeled real world 
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situation. Commonality and variability are competing concerns and its hard to 
find a proper balance between them by approaches that (a) model the real world 
from the scratch and then (b) try to reuse the common aspects in such upfront 
design models. The forces in the steps (a) and (b) can normally not be well in- 
tegrated. We rather propose in this work to model only the interfaces and keep 
them variable. Techniques for “programmable interfaces” let us flexibly glue the 
application parts together. 

There are recurring ways for finding good abstractions and partitionings. 
“Good” means that they provide a tenable amount of commonalities to let us 
understand the problem and produce long-lasting software, but still enable us 
to easily introduce (expected and unexpected) changes. Such patterns of or- 
ganizing abstraction around commonalities and variations are popularly called 
“paradigms” . In software engineering a paradigm is a set of rules for abstrac- 
tion, partitioning, and modeling of a system. E.g., the object-oriented paradigm 
structures the design/program around the data, but focuses on behavior [23]. It 
allows us to introduce variations in data structures/connections and algorithm 
details. Each paradigm has a key commonality and variation. 

If we implement a system, we have to deal with a broad variety of paradigms. 
Coplien [3] discusses the need for multi-paradigms. In fact nearly any good real 
world software system is designed and implemented using multiple paradigms, 
simply because nearly no complex real situation exists, that can be described 
with one paradigm sufficiently. E.g., in nearly every large C-|— I- program a mix- 
ture of object-oriented, procedural, template, and various outboard paradigms 
exists. Here, outboard paradigm [3] means a paradigm that is not supported by 
the programming language itself, but by a used technology, like the relational 
paradigm adopted from a relational database. 

In the focus of this paper are language constructs and concepts for design 
and implementation that overcome current problems of the component- and 
object-oriented paradigms and their integration. Firstly, we will discuss these 
paradigms and their current integration problems. Afterwards we present some 
language concepts of the language XOTcl: Firstly we will discuss concepts which 
can be mapped manually to current mainstream languages, then we will present 
some interception techniques that are missing in current mainstream languages. 
Finally we will generalize our approach and compare to related work. 

2 Combination of Component- and Object-Oriented 
Paradigm 

2.1 Component- Oriented Paradigm 

The very idea of component-based development is to increase productivity of 
building software systems, by assembling prefabricated, widely-used components. 
Components are self-contained, parameterizable building blocks with explicit 
interfaces. Component-based development aims at the replaceability of compo- 
nents and the transferability of components to a different context, thus enabling 
component reuse. 




116 



Michael Goedicke, Gustaf Neumann, and Uwe Zdun 



The idea of the component-oriented key abstraction is not new. E.g., in many 
large C systems self-contained components (or modules) that can be accessed via 
an explicit API can be found. Component-based development, as it is proposed 
today, mainly adds interface definition languages or other means to enforce that 
all component accesses conform to the component interface, platform and/or 
language independence, support for distribution, and accompanying services. 

Current component approaches, such as component frameworks in scripting 
languages, like Tcl [19], or the component models of popular middleware ap- 
proaches, such as CORBA, Java Beans, or DCOM, induce mainly a black-box 
component approach. Unfortunately often there are several factors for develop- 
ment organizations that drive them not to adopt the component-based approach 
for components developed by a third-party or even by a different in-house devel- 
opment team. Often used arguments are, that internals of a black-box component 
can not be changed, therefore, reaction on business process changes can become 
more difficult. Generally, hugs in the component are harder to fix, because it is 
hard to build a work-around for a bug, that is not located in your own code. The 
organization relies on the ability and will of the component developer to fix the 
bug. Moreover, using black-boxes often means that the development team looses 
expertise on component’s domain. 

These factors can be observed in many real world applications and they 
apply not only for black-boxes, but also (to a smaller degree) to components 
with available source code. The component abstraction seeks for building blocks 
that can be produced and maintained separately from the systems they are used 
in. Variations have to be treated separately and are mainly introduced by means 
of parameterization. The key problem of component-based software engineering 
is: On the one hand, components aim at extracting the commonalities to a level, 
where we can use them as prefabricated building blocks. On the other hand it is 
hard to maintain and to cope with changes in a piece of software without access 
to internals. Since parameters are the main variation technique of black-box 
components, the changeable parameters have to be foreseen by the component 
developer. But in reality often the requirement changes are not foreseeable at 
component development time. 



2.2 Object-Orientation and Components 

Object-orientation is a paradigm sharing properties with component-based ap- 
proaches. Many component-based approaches are implemented using objects. 
Object-orientation arranges structures around the commonalities of data, but 
focuses on behavior. Traditional object-oriented programming and design maps 
entities of the modeled real world to a single programming language/design 
construct: the class. Object oriented design expresses computational artifacts 
through a mapping onto several classes and their relationships. In most object- 
oriented approaches, these relationships are association/aggregation (most often 
both are expressed through the same language construct), and inheritance or 
extension. 
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Object-orientation promises separation of the involved orthogonal concerns 
through encapsulation and class inheritance. The abstraction into general parts 
with inheritance or delegation in object-based system should help us to concen- 
trate on common and special properties at different times. These abstractions 
also promise to gain modularity and to anticipate changes by designing general 
modules, that can be specialized in different ways and that support incremental 
extensions. 

These promises were only partially achieved by traditional object-oriented 
approaches. Studies of the amount of reuse gained through object-orientation 
indicate that reuse is much smaller than expected in its early promises. E.g., 
Ousterhout [19] points out on basis of empirical evidence that the reuse of com- 
ponents - as they are used in scripting languages - is by far higher, than the 
reuse gained solely by object-orientation. We believe a main reason for this di- 
vergence is that large object-oriented frameworks tend to require an intimate 
knowledge of the framework’s internals. 

The component-oriented key abstraction of consequently exploiting parame- 
terizable black-boxes is more suitable for reuse. But when taking a closer look at 
the success factors of scripting languages, one can observe that they combine the 
components with a highly-ffexible glue language. Object-orientation especially 
helps us to understand structural complexity and to make it explicit by architec- 
tural means. These architectural means can be extremely valuable in complex 
design situation in the glue language (see [18]), because a relatively small glue- 
ing application can become very expressible and complex through the number of 
involved components. And object-oriented language constructs can also be valu- 
able in the internal design of the self-contained components. Our approach relies 
on the two-level concept of scripting languages, like Tcl. We distinguish into 
reusable, self-contained components, mostly written in a system language, and 
a high-level, object-oriented scripting language that combines these components 
flexibly. 

Complex design problems are a focus of object-oriented approaches, but a 
weak point of component combination with a scripting language, like Tcl. Object- 
oriented design patterns capture the practically successful solutions of the field 
of object-orientation. Our research on language support for design pattern, as 
for instance in [14], has shown that pattern variations cannot be described solely 
through parameterization. Reusable pattern implementation variants have to 
be fitted to the current context, especially when patterns are used in the hot 
spots [20] of software systems. These parts of the application - where an elegant 
and sufficient solution requires variabilities beyond pure parameterizations - are 
the parts that are hard to cover with the component-oriented abstraction, since 
its key variability is parameterization. The combination of the two paradigms 
with high-level design/implementation language functionalities lets the object- 
oriented constructs cover the weak points of the black-box component approach 
and vice versa. 
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3 Components and Component Configuration in XOTcl 

Extended Object Tcl (XOTcl) [18] (pronounced exotickle) is an object-oriented 
extension of the language Tcl. Scripting languages gain flexibility through lan- 
guage support for dynamic extensibility, read/ write introspection, and automatic 
type conversion. The inherent property of scripting languages such as Tcl is 
that they are designed as two-level languages, consisting of components written 
in efficient and statically typed languages, like C or C-|— k, and of scripts for 
component glueing. 

Our assumption is that just “glueing” is not enough. XOTcl enhances Tcl 
with language constructs giving architectural support, better implementation 
variants, and language support for design patterns, and explicit support for com- 
position/decomposition. All object-oriented constructs are fully introspectable 
and all relationships are dynamically changeable. XOTcl offers a set of basic 
constructs, which are singular objects, classes, meta-classes, nested classes, and 
language support for dynamic aggregation structures. Furthermore, it offers two 
message interception techniques per-object mixin and filter, to support changes, 
adaptations, and decorations of message calls. 

In XOTcl a component is seen as any assembly of several structures, like 
objects, classes, procedures, functions, etc., to a self-contained entity. Compo- 
nents are conveniently packed into packages that can be loaded dynamically. A 
component can also consist of a C or C-I--I- extension of Tcl. Each component 
has to declare its name and optional version information with Tcl’s package 
provide with the following syntax: 

package provide componentName fversion? 

The system automatically builds up a component database. With package require 
an XOTcl program can load a component dynamically with a name and optional 
version restrictions at arbitrary times, package require has nearly the same syn- 
tax: 



package require componentName fversion? 

Components expose an explicit interface that can be used by other programs 
without interfering with the components internals. But still we have to inte- 
grate the components with the application and make the component internals 
adaptable and dynamically fit-able to a changing application context. 

3.1 Component Wrapping 

Component wrappers can wrap black-box components written in various lan- 
guages and structured with multiple paradigms. The component wrappers are 
object-oriented Wrapper Facades [21] that shield the components from direct 
access (see Figure 1). Note, that often a set of interacting component wrappers 
has to be used to wrap a complex component properly. Above the component 
wrapper layer a set of implementation objects define the hot spots of the de- 
sign. All objects (including the component wrappers) can exploit the dynamic 
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and introspective language functionalities of XOTcl. Since a black-box compo- 
nent is never accessed directly, but always with the indirection of the component 
wrapper, we gain a central place, that is a proxy or placeholder for a component. 
The component wrapper is a white-box for the development team of the applica- 
tion. Here, changes can be applied centrally and adaptations can be introduced 
without affecting the components’ internals. 




Figure 1. Integration of Components and Objects through Component Wrap- 
pers. 



Generally each component wrapper is implemented with an abstract interface 
and a concrete component wrapper implementation (see Figure 2). Clients use 




Figure 2. Class-Based Component Wrapper Interface. 



the components as Strategies [4] to make components easily exchangeable by 
providing a new concrete component wrapper and by dynamically changing to a 
new Strategy. The concrete component wrappers forward the received messages 
as Wrapper Facades [21] to the components that implement the functionality. At 
the connection between client and component wrapper we can easily enhance the 
functionality of the component with Decorators [4]. At the connection between 
component wrapper and component we can use Adapters [4], e.g. to perform 
interface adaptations. 
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3.2 Export /Import Component Configuration 

The implementation of the component’s functionality (e.g. in C or C++) is inte- 
grated into XOTcl with Tcl commands (see Figure 3). A component explicitly 
defines its export by explicitly defining a set of Tcl commands (function names 
with argument lists). These commands can be mapped onto one (or more) wrap- 
per objects, that configure the component usage and adapt the Tcl Commands to 
an object-oriented interface. The component wrapper explicitly declares which 
of the exported methods are the import of this component usage. This way the 
component’s client defines the required interface that an implementing compo- 
nent has to conform to. The component implementation can be replaced by any 
other implementation that conforms to the required interface. Finally, the actual 
implementation objects, which are using the component, call the methods of the 
component wrapper. 
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Figures. Three-Level Component Configuration with Explicit Export/Import. 

The implementation objects can be used to build an application or a new 
component. If a new component is built from existing components, it can export 
an interface through a component wrapper consisting of XOTcl methods. But, 
since any Tcl program can be embedded in a C program, a new component can 
also export C functions (which can be used by any C program). 

The component concept relies at runtime on the concept of component con- 
figuration [5] . The first configuration step maps a C library component with an 
interface design into the scripting language. Then this functionality is imported 
and adapted by the component wrapper. Finally, the implementation objects 
use the adapted import in their application framework. Each configuration step 
allows us to actualize the configurations with different implementations that 
conform to the interfaces. The integration of C components is presented in the 
upper half of Figure 3. 

The general technique of applying an Object System Layer to a base language 
and to implement the components with an object-oriented implementation is 
presented below. This technique is used in various languages and applications 
and is documented as the Object System Layer architectural pattern [12]. 
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Component configuration ~ as used in this work - is the runtime technique of 
combining components. In XOTcl each component configuration can be changed 
dynamically at arbitrary times. The component import interfaces can be dynam- 
ically fitted to the new context. In order to keep track with this runtime flexibility 
an important functionality of the XOTcl language is introspection. It allows us 
to query the import interface for method names, argument list, and method im- 
plementations. The currently configured components can be queried to trace the 
components, their configuration, and the used interfaces at runtime. Runtime 
inspection tools can be written with a few lines of code. 

4 Interception Techniqnes for Flexible Component 
Wrapping 

The techniques discussed so far can (mainly) be implemented in any object- 
oriented design/programming language. The only difference of using XOTcl is 
that XOTcl language supports the discussed component concept already, i.e., 
it offers dynamic package loading mechanisms, language support for dynamic 
aggregation, dynamics and introspection in all language constructs, etc. In other 
languages we have to program the concept implementations by hand. But im- 
plementing flexible component wrappers solely with class constructs has several 
disadvantages: 

— Transparency: The client should use the abstract interface without knowl- 
edge of concrete implementation details. The component wrapper should not 
appear to be scattered over several implementation objects. 

— Concerns that cross-cut the component wrapper hierarchy. Often a complex 
class hierarchy is necessary to implement component wrappers sufficiently. 
E.g., most widget sets offer widgets as C or C-I-+ components. In order 
to compose compound widgets out of simpler widgets, we may need object 
hierarchies as in the Composite pattern [4]. Concerns that are of a broad 
structural size and that cross-cut the hierarchy, such as painting of the whole 
compound widget, conventionally have to be programmed by hand. 

— Object-specific component wrapper extensions or adaptations: Often adapta- 
tions have not to be performed for all objects of a certain component wrapper 
type, but only for one object. We should be able to object-specifically en- 
hance components, without sacrificing the transparency. The intrinsic com- 
ponent wrapper implementation and the implementation of extension/adap- 
tation parts should remain decomposed. 

— Coupling of Component and Wrapper. Component and component wrapper 
should appear as one runtime entity to clients, but they should be decom- 
posed in the implementation. 

— Dynamics in Component Loading: Components should be dynamically load- 
able, replaceable, and removable. 

— Runtime Traceability: Components are loaded (possibly dynamically) into 
the system. To know which components are already loaded, the connections 
between wrapper and component should be traceable at runtime. 
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In this section we will briefly explain two interception techniques of XOTcl, that 
overcome these problems by flexible adaptation of the component wrapper calls 
to the concrete implementation. Both are transparent for the client. The per- 
object mixin implements concerns that are object-specific extensions, while the 
Alter implements concerns that cross-cut class hierarchies. Filters and per-object 
mixins form runtime traceable entities with the intercepted objects at runtime, 
but are decomposed in the implementation. 

4.1 Per-Object Mixins for Object-Specific Component Wrapper 
Extensions 

A per-object mixin [13] is a language construct, that enhances a single ob- 
ject with a class that is object-speciflcally mixed into the precedence order 
of an object in front of the precedence order implied by the class hierarchy. 
Through object-specific, transparent, and dynamic interception of the messages 
that should reach the object, object-specific pattern variants [15] and object- 
specific roles [13] can be implemented conveniently. Per-object mixins allow us 
to handle orthogonal aspects not only through multiple inheritance, but since 
they are themselves classes and use class inheritance, they co-exist with the 
object’s heritage order. 




Figure 4. Request Logging/ Access Control with Per-Object Mixins. 



In Figure 4 we can see an example of a per-object mixin. An abstract class 
Request has two subclasses, one handling HTTP requests and one for FTP requests. 
The class definitions may look like: 

Class Request ;# Abstract class definition 

Request abstract instproc open {} ;# Abstract method 

Class HTTP -superclass Request 
HTTP instproc open {}■ { 

} 

Class FTP -superclass Request 

HTTP and FTP objects are Wrapper Facades [21] to components that implement 
the actual requests as black-boxes. Orthogonal to the tasks of a requests are the 



;# HTTP class definition 
;# Method definition 
;# Method forwards to HTTP 
component 

;# FTP class definition 
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tasks of request logging, which can operate on both mentioned request types. 
In many cases only certain specified request objects should be logged, as in the 
example httpl. 

We do not want to interfere with the internals of the components that im- 
plement the requests in order to gain request logging. Therefore, a solution 
with single or multiple inheritance would not suffice, because it would either 
make all requests logged or create unnecessary intersection classes [13], like 
LoggedHttpRequest and LoggedFtpRequest. A solution with a reference from a 
logging object, as in the Decorator pattern [4], would require the client to main- 
tain a reference to the perhaps volatile logging object and, therefore, it would be 
not transparent to the client. A solution with a reference to a logging object, as 
in the Strategy pattern [4], would not be transparent to the request object and 
unnecessarily interfere with the internals of the component wrapper. Both solu- 
tions suffer from the fact that - from the viewpoint of the client - one conceptual 
entity is split up into two runtime entities. 

The solution with the per-object mixin, as in Figure 4, does not suffer from 
any of these problems. It attaches the role of being a logged request and an 
access control mechanism as a second orthogonal aspect object-specifically to 
the request object, either in Decorator or Strategy style (as required). The ac- 
cess control mechanism is actually performed in an imported component, while 
the rather simple task of logging is handled by the mixin class. The per-object 
mixin is transparent to client and request object. The logged request appears 
as one conceptual entity to the client. There is only one object httpl that can 
be accessed and it always has the same intrinsic class HTTP. But still logging 
and request tasks are decomposed into different classes and can be dynamically 
connected/disconnected. Per-object mixins can be attached in chains and spe- 
cialized through inheritance. The per-object mixin solution may look like: 

HTTP httpl ;# Instantiation of httpl object 

Class LoggedReq ;# Logged request class definition 

LoggedReq instproc open {}■ { 

# logging implementation 
next 

} 

Class AccessControl 

httpl mixin {AccessControl LoggedReq} ;# Mixin registration 

LoggedReq and AccessControl are ordinary classes of XOTcl. As an example 
method, we define a method open for LoggedReq that logs all open calls and 
forwards the message afterwards with the next language primitive to the next 
mixin or the actual method implementation. We dynamically register the mixin 
classes for httpl. 
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4.2 Filters for Cross-Cutting of Class Hierarchies 

A second interception technique, called filter [14], is able to operate on a class 
hierarchy, instead of a single object (as the per-object mixins). A filter is a 
special instance method registered for a class C. Every time an object of class C 
or one of its sub-classes receives a message, the filter is invoked automatically. 
A prominent concern for usage of filters is to implement larger artifacts of the 
modeled world, like object-oriented design patterns, as instantiable entities of the 
programming language. In [14] we show how to express such concerns through a 
meta-class with a filter. The filter operates on all classes derived from the meta- 
class. Filters can be defined once and can then be registered dynamically. One 
can introspect the filters that are registered for a class. All filter invocations are 
transparent for clients. 

As an example for filters, we present the implementation of a Composite pat- 
tern [4] variant in a reusable component. The context of the composite pattern 
is to build up an object tree structure and to derive a set of classes from an 
abstract component type. E.g., a composite widget component Canvas can ag- 
gregate other widgets. This way we can build up compound widgets. All widgets 
confirm to the same abstract widget component interface. A recurring problem 
of such structures is that leaf object, like button widgets, are not allowed to 
aggregate other objects. The solution to the problem in a purely class-based 
environment is, that only aggregating composite objects contain an aggrega- 
tion relationship. Leaf objects, like buttons in the widget component example, 
have no children. An intrinsic property of composite hierarchies is that certain 
operations on the root component, as in the widget example a painting of the 
compound widget, have to be propagated through the object tree. Composite 
objects forward these operations to all their children, while leaf objects only 
execute the operation, but do not forward. 

There are several problems that can be identified with the class-based im- 
plementation of the pattern. Concerns that cross-cut the composite hierarchy, 
like the forwarding of messages or the life-time responsibility of the whole for 
its parts (in the widget example: if a top-level widget is destroyed, all the con- 
stituent components have to be destroyed), are not expressed properly, since 
their semantics are not handled automatically. There is a certain implementa- 
tion overhead, e.g. due to unnecessary forwarding of messages. The pattern as a 
conceptual entity of our design is neither traceable in the program code nor at 
runtime, but it is scattered across several implementation constructs. This vari- 
ant of pattern implementation can hardly be reused, when the implementation 
language lacks proper (dynamic) means to fit a general variant implementation 
to a new context. 

The implementation with a filter, a meta-class, and dynamic object aggrega- 
tion, as in Figure 5, does not suffer from these problems. The filter is stored in 
a dynamically loadable component that contains a meta-class with a filter. The 
filter implements the reusable pattern implementation variant. Classes, like the 
widget component, that are superclasses of composite types are of the Composite 
meta-class type. Automatically they get the filter registered. The filter acts on 
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Figure 5. Composite Pattern with Filters/Dynamic Object Aggregations. 



the whole composite hierarchy and implements the concerns that cross-cut the 
hierarchy. Here, the forwarding of composite messages on a set of registered op- 
erations, like paint, is such a concern. The filter in the meta-class is transparent 
to clients. The compound widget appears as every other ordinary widget. The 
dynamic object aggregation language construct automatically handles the aggre- 
gation issues, like assurance of the tree structure and the life time responsibility 
of wholes for their parts. 

The class definition of the Composite meta-class firstly has to define the meta- 
class. The composite filter is an ordinary instance method that determines the 
called method with an introspection option. It calls the message on all children 
with a loop and then on the current object. The component may look like: 
package provide Composite 0.8 



Class Composite -superclass Class ;# Meta-class definition 

Composite instproc compositeFilter args { ;# Composite filter method 



set r [[self] info calledproc] 
foreach child [[self] info children] {. 

eval $child $r $args 

} 

return [next] 



} 



;# Determine called operation 
;# Loop over all children 
objects 

;# Forward message to children 

;# Call message on 
‘ self ’ -object 



We can load the Composite implementation from a component with package 
require. Afterwards we define the class hierarchy. WidgetComponent is defined by 
the meta-class Composite and automatically handles the forwarding of messages 
for all sub-classes transparently, 
package require Composite 



Composite WidgetComponent -superclass Widgetinterf ace 
Class Button -superclass WidgetComponent 
Class Canvas -superclass WidgetComponent 
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5 Components for Development of Flexible Software 
Architectures 

The outcome of software development is a piece of software, a sustainable intel- 
lectual structure, which manifests the results of the design. Often it is handed 
over from the development team to the maintainers. The ability to modify or 
to understand the software especially for a person that was not involved in the 
development details, depends on the software architecture. 

There are several properties that we expect from a “good” software architec- 
ture: It should be flexible, evolvable, understandable, predictable and maintain- 
able. We expect the architectures to offer a significant amount of code reuse to 
speed up the development process and to achieve more reliable software systems. 
Certainly, the systems should be highly efficient. In summary we can make the 
following assessments on the architectural impact of the approach discussed in 
this paper: 

— Heterogeneous, Multi- Paradigm Black-Box Components: Black-box compo- 
nents from various languages, like Tcl, XOTcl, C, or C-|— k, are reused. The 
components are implemented with the most suitable paradigms. Unfortu- 
nately flexibility and evolvability of our software architectures suffer from 
our inability to change or react on problems of the components internals. 
In this paper we have discussed two approaches - scripting and high-level 
object-orientation - to overcome these problems. 

— Object-Oriented Scripting Language as a Component Clue: Scripting lan- 
guages combine components flexibly, by means of a highly flexible, intro- 
spective, and dynamic glueing language. But scripting languages are not 
very suitable to express the complexity of large application frameworks (see 
[18,14]). Their original language design aims at smaller applications, partly 
because of runtime efficiency. However, time critical parts can always be put 
into components written in more efficient languages, like C. Still complex 
scripting applications, like several compound widgets in TK, were very slow 
in the early days of Tcl/TK. But nowadays CPU speed allows us to build 
very complex scripting applications without a reasonable speed penalty. 

In contrast, the combination with object-orientation gives us architectural 
support for composition/decomposition. The hot spots of the application, 
which are expectable changing parts, are kept in the high-level object-orien- 
ted language with its dynamic and introspective language means. Design 
experience of the object-oriented community with complexity of applications 
and with introducing flexibility in framework hot spots, helps us to make the 
component wrappers easily (ex-) changeable and evolvable. 

— Component Wrappers: Object-oriented component wrappers integrate the 
components into the scripting language (if necessary) . With the set of com- 
ponent wrappers a component’s client explicitly defines its import from the 
component. In turn, the component explicitly defines its export through the 
Tcl wrapper. In a three-level process of configuration we actualize the inter- 
faces with concrete implementations. These can be exchanged against other 
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implementations transparently, what leverages evolvability and flexibility of 
our architectures. The implied indirection fosters understandability, since 
we can understand components and clients independently. The component 
wrappers are introspect able white-boxes to the application. Often changes 
in the component wrapper ~ without interference with the component’s in- 
ternals - are sufficient to cope with new requirements for a component. 

— Interception Techniques: Object-oriented interception techniques, like filters 
and per-object mixins, enable us to deal with concerns that are hard to ex- 
press with current object-oriented constructs. They are especially valuable 
on the component wrapper, since they allow us to transparently and dy- 
namically introduce multiple views onto a component implementation (at 
runtime) . 

To back up the results in this work we have provided two case studies of 
systems build with the techniques described in this paper. In [16] we describe 
our web server implementation in XOTcl. We have compared efficiency with the 
pure C-based Apache web server. In the worst case our implementation was 25% 
slower than Apache, in some cases, our implementation was faster. In [17] we 
present a high-level framework for XML/RDF text parsing and interpretation. 
Again we have performed speed comparisons with implementations in Java and 
C. The Java based implementation was 2-4 times slower than our implementa- 
tion in the scripting language (using off-the shelf C components), the pure C 
implementation was only 1.5-3. 5 times faster than the scripting implementation. 

6 Related Work 

In [22] the (mainly black-box) component models of current standards, like 
CORBA, COM, or Java Beans are discussed. These approaches offer the bene- 
fits of black-box component reuse, but have problems, when the internals of a 
component have to be changed or adapted. Currently none offers an integrated 
concept for component reuse and adaptation, that solves all the problems raised 
in this paper, but all approaches have extension in this direction. 

Java Beans implement limited dynamic loading and reflection abilities, but 
not all interesting data can be introspected (e.g., the important information on 
caller/callee of a call can not be retrieved automatically). Java Beans offer a dis- 
tinct component model. But Java Beans offer only the Java Native Interface for 
integration of components written in other programming languages. Java does 
not support powerful language means for configuration/adaptation of compo- 
nents. 

In [7] an interception system for COM objects that can intercept object 
instantiations and inter-object calls is presented. COM is a binary object model 
and the approach relies on direct manipulation of the function pointers that call 
the methods. In contrast to the approaches in this paper the approach is a very 
low-level approach and does not support suitable introspection mechanisms. 

Orbix filters [8] (and similar techniques in other ORBs) implement limited 
interception abilities. They do only operate on distributed method calls and do 
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not offer sophisticated introspection techniques. A more general form of such 
abstractions of the message passing mechanisms in distributed systems are com- 
position filters [1]. Abstract communication types are used as first-class objects 
that represent abstractions over the interaction of objects. They encapsulate 
and enforce invariant behavior in object communications, can achieve the reduc- 
tion of complexity of object interactions, and can achieve reuseability of object 
interaction forms. 

The new CORBA 3.0 standard specifies a component model that is based 
in part on the Java EJB component concepts, but goes beyond that, by pro- 
viding the component model to work with various languages. The new CORBA 
standard includes a scripting language specification, which is a framework to 
integrate scripting languages with a CORBA mapping. Interestingly, the goals 
for primary applications of the scripting language specification are the same 
issues, for which we have given architectural language support in this work, 
i.e., customizations of components, legacy wrapping, a service-based callback 
architecture, and flexible component glueing. It is likely that XOTcl and the 
techniques discussed in this work would ease it to reach these goals (though the 
specification is still in an early state). 

A role, as in [11], is used to express the combination with extrinsic proper- 
ties that can be dynamically taken/abandoned. The approach does not define a 
comprehensive read/write introspection mechanism. It does not provide an ab- 
straction at a broader structural size, as the filter that applies a role on a whole 
class hierarchy. 

Aspect-oriented programming [10] is a programming technique for decompos- 
ing concerns into aspects, that have to be coordinated with other concerns across 
component boundaries. Aspects cross-cut component boundaries, while compo- 
nents are characterized by being cleanly encapsulated. An “aspect weaver” (a 
kind of a compiler) weaves components and aspects together. The approach does 
only introduce limited (dynamic) changeability through re-weaving, but it can 
express concerns that cross-cut several components properly. In contrast to our 
approach, aspect-orientation is not inherently a black-box approach, but requires 
knowledge of the component’s internals to build useful aspects. 

Meta-object protocols, as in [9], divide a system into meta-level and base- 
level. Meta-level objects impose behavior over base-level objects. Generally meta- 
object protocols are low-level, but powerful; they can achieve reflection, dynam- 
ics, and transparency. Our approach provides meta-classes with similar abilities, 
but most often the interceptors are more powerful, easier composable, and pro- 
vide more introspection facilities. 

Bosch proposes in [2] a component adaption technique based on layers, which 
is similar to the presented interceptors: it is also transparent, composable and 
reusable, but it is not introspective, not dynamic and a pure black-box approach. 
Layers are given in delegating compiler objects, that are statically defined be- 
fore compilation time. This approach can be hardly used for expressing runtime 
dynamics in component composition, since changes in layer definitions require 
recompilations . 
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Subject-oriented programming models [6] offer different views for a client 
onto a concern. Classes can be composed with composition rules. In contrast to 
composition with interceptors, it does not provide introspective and dynamical 
runtime composition, but only by a tool called subject compositor. Subject- 
orientation (and subsequent approaches) address composition of system parts 
with extensions and multiple views, thus it helps to overcome some of the prob- 
lems of (descriptive) component composition. Central runtime problems of com- 
bining components, like adapting components without interference with compo- 
nent internals at different granularities, component introspection/runtime trace- 
ability, or dynamic component loading/unloading are nearly not addressed. 

7 Conclusion 

We have addressed the issue that software architectures should be build with 
reusable (off-the-shelf) black-box components, but should also exploit the char- 
acteristics of white-box object-oriented frameworks regarding evolvability and 
flexibility. We have presented a practical approach that integrates such black- 
boxes, written in several languages. An object-oriented scripting language serves 
as a component glue with explicit export/import interfaces and as a central place 
to introduce changes into the hot spots of the architecture. To overcome several 
problems of object-orientation, the language offers interception techniques, that 
are valuable for component composition and adaptation. All presented tech- 
niques can be used in various other languages through explicit programming 
by hand. Nevertheless, (runtime) language support for introspection, dynam- 
ics, component composition/decomposition, and interception techniques is use- 
ful, since it leads to shorter, more elegant, and less error-prone solutions. Since 
XOTcl is itself a C library it can be embedded in any C or C-I-+ application as 
a distinct component glueing language. 

XOTcl can be downloaded from http://www.xotcl.org. 
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Abstract. The behavior of a system comprised of collaborating components tend 
to be difficult to analyze, especially if the system consists of a large number of 
concurrently operating components. We propose a scenario-based approach for 
analyzing component compositions that is based on Use Case Maps (UCMs), but 
is extended with a few additional constructs for modeling component interfaces 
and connections. UCMs provide a high level, behavioral view on a system that 
is easy to comprehend by humans. However, UCMs do not have well-defined se- 
mantics. For this reason, UCMs are augmented with formal component interface 
specifications as used in the concurrent, object-oriented programming language 
BCOOPL. The combination of UCMs and BCOOPL interface specifications en- 
ables formal analysis of component compositions. This involves two steps. In 
the first step, UCMs and BCOOPL interface specifications are translated into a 
BCOOPL program. In the second step, the interactions between components are 
analyzed for system properties like deadlock and reachability. An important result 
of the combination is that the complexity, which arises when concurrently col- 
laborating components are brought together, is tamed by considering only those 
usages of components that are actually specified in UCM scenarios. 



1 Introduction 

A high level design notation targeted to model component compositions should serve 
two purposes. On the one hand, the notation should be easy to understand by humans, 
in particular, designers. A designer should gain insight in the behavioral aspects of the 
system as a whole almost effortlessly. On the other hand, the notation should be precise 
enough to reason about compositions. 

The behavior of a system can be defined with formalisms like temporal logics and 
traces of externally observable events, which can be represented formally and graph- 
ically with notations such as Petri nets [14] and StateCharts [12], With suitable ab- 
straction mechanisms we can then define the components and events of interest to gain 
insight in the overall behavior. However, this approach does not give the big picture. 
What is needed is a notation with which designers can reason about the behavior at 
a high abstraction level. This purpose can be fulfilled with a scenario-based technique 
called Use Case Maps (UCMs) [2] as will be shown in a couple of examples. One of the 
strong points of UCMs is that they can show multiple scenarios in one diagram and the 
interactions amongst them. This allows a designer to reason about a system as a whole 
instead of focusing on details. 
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Real-size systems tend to be hard to analyze. Especially if a system is composed 
of concurrently running components, the number of possibihties to consider explodes 
exponentially. The complexity, which is caused by non-deterministic behavior, can be 
tamed by considering only those scenarios that can realty happen. In particular, a com- 
ponent may provide a number of scenarios (behaviors) of which only a subset is actually 
used in a system. Thus, the scenarios provided by UCMs gives us a handle to prune aU 
non-occurring behaviors of components beforehand. 

UCM is an informal notation. This apparent shortcoming is precisely the reason why 
UCM can be used at high abstraction levels; low-level details are simply not part of the 
notation. By augmenting UCM with component interface specifications, it is possible 
to reason about component compositions formally. For instance, we will show in an 
example how the presence or absence of deadlock in a system can be detected. Another 
apphcation area is to prove reachability properties, i.e., to verify whether a system can 
achieve certain goals or not. 

In this paper, we combine UCMs with BCOOPL (Basic Concurrent Object-Oriented 
Programming Language) interface specifications, which not only detail how operations 
should be invoked, but also when operations may be invoked and the parties that are 
allowed to do so [8]. The temporal orderings of operation invocations provide a good 
starting point for analyzing whether a system comprised of components violates the 
imposed orderings or not. Apart from the aspects mentioned above, BCOOPL supports 
other language features that are useful for component-oriented programming, such as 
the built-in support for the Observer design pattern for minimizing the coupHng between 
components, delegation for supporting black-box composition, concurrency for deahng 
with distributed computing transparently, and component access control for addressing 
security issues. These language features allow components to be specified at a higher 
abstraction level with respect to general purpose languages hke C++ and Java. In fact, 
BCOOPL can be regarded as a design language with which executable designs can be 
specificed. The combination of UCMs and BCOOPL interface specifications can be 
translated unambiguously into a BCOOPL program. This BCOOPL program can then 
be subjected to analysis in order to prove certain system properties such as deadlock 
and reachabihty. 

This paper is organized as follows. We start with an overview of UCM and 
BCOOPL. Next we introduce a running example to exemplify the two steps involved in 
analyzing a system. This is followed by a detailed explanation of the two steps: trans- 
lating a specification into a BCOOPL program and analyzing the resulting program for 
system properties. We end this paper with a discussion and concluding remarks. 



2 Use Case Maps and BCOOPL Preliminaries 

The material in this paper builds on previous work in which a grey-box approach to 
component specification is argued [6]. In principle, a black-box approach to compo- 
nent deployment should be favored. In practice, however, we require information that 
cannot be described solely in terms of externally visible properties of components. For 
instance, non-functional properties (e.g., space and time requirements), environmental 
dependencies, and variation points (e.g., places where a component may be adapted or 
extended) do require insight in the internal construction of a component. The combina- 
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tion of UCMs and BCOOPL interfaces gives us the opportunity to document intra and 
inter component behavior at a high, but formal abstraction level. 



2.1 U.se Case Map.s 

A UCM is a visual notation for humans to use to understand the behavior of a system 
at a high level of abstraction. It is a scenario-based approach showing cause-effects by 
travehng over paths through a system. UCMs do not have clearly defined semantics, 
their strong point is to show how things work globally. 

The basic UCM notation is very simple. It is comprised of three basic elements: re- 
sponsibihties, paths and components. A simple UCM exempHfying the basic elements 
is shown in Figure 1 . A path is executed as a result of the receipt of an external stimulus. 
Imagine that an execution pointer is now placed on the start position. Next, the pointer 
is moved along the path thereby entering and leaving components, and touching re- 
sponsibihty points. A responsibihty point represents a place where the state of a system 
is affected or interrogated. The effect of touching a responsibihty point is not defined 
since the concept of state is not part of UCM. lypically, the effects are described in 
natural language. Finally, the end position is reached and the pointer is removed from 
the diagram. A UCM is concurrency neutral, that is, a UCM does not prescribe the 
number of threads associated with a path. By the same token, nothing is said about the 
transfer of control or data when a pointer leaves one component and (re-)enters another 
one. The only thing that is guaranteed is the causal ordering of executing responsibihty 
points along a path. However, this is not necessarily a temporal ordering, the execution 
of a responsibihty point may overlap with the execurion of subsequent responsibihty 
points. 



Start 



Components 















\ rl 










\ r2 
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Pointer 
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Fig. 1. UCM basic elements. 



The UCM notation is quite rich, and only a smaU subset is used in this paper. In 
Figure 2, two frequently used UCMs constmcts are shown. The AND-construct is used 
to spawn multiple activities along parahel paths. When a pointer reaches an AND-fork, 
this pointer is removed from the diagram and replaced by two pointers at the beginning 
of the parallel paths. An AND-join acts as a synchronizahon mechanism. When both 
pointers have reached the AND-join, they are replaced by a single pointer and execution 
is continued along the path fohowing the join. The OR-construct should be interpreted 
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as a means to express multiple scenarios in a single diagram. It states that multiple 
scenarios are comprised of identical paths. Therefore, it is not necessary to specify 
conditions detaiUng the path to be followed at an OR-fork. 



AND-fork AND-join 




OR-fork ' . OR-join 




Fig. 2. Addidonal UCM notation. 



2.2 BCOOPL 

BCOOPL is a small, concurrent object-oriented programming language specifically de- 
signed to support component-oriented programming. BCOOPL has a long research his- 
tory. It roots can be traced back to path expressions [4], and the concurrent object- 
oriented programming languages Procol [15] and Talktalk [1]. One of the strong points 
of BCOOPL is the built-in support of design patterns catering for component-oriented 
programming (see [10] for a pattern catalog). In particular, BCOOPL supports the Ob- 
server, the Mediator and the Bridge design patterns directly. The Observer and Bridge 
pattern are particular useful for specifying stand-alone components, while the Medi- 
ator pattern is used to mediate interactions between those components [7]. Other de- 
sign patterns frequently used in components, such as the Facade and the Proxy, can be 
implemented relatively easily in comparison with more traditional object-oriented pro- 
gramming languages like Java and C-t-i-. A detailed account of BCOOPL and some of 
its apphcation areas can be found in [8], which also addresses implementation issues. 

2.2.1 Core Language Features BCOOPL is centered around two concepts: inter- 
faces and patterns. An interface defines the operations that must be implemented by an 
object that conforms to that interface. By adhering to the principle of programming to an 
interface, a certain amount a flexibihty is added to a system since new implementations 
can be provided without breaking existing code. A BCOOPL interface is specified as an 
augmented regular expression over operations. It not only describes how an operation 
can be invoked, but also when and by whom. 

Class and method definition have been unified in patterns and sub-patterns. The 
term pattern has been borrowed from the object-oriented programming language Beta 
[13]. The idea is that objects are instantiated from patterns and behave according to 
the pattern definition. A pattern describes the allowed sequences of primitives to be 
executed by an object after a message has been received in a so called inlet, which 
is imphcitly defined in a pattern definition. It is specified as a regular expression over 
primitives using the same operators as in interface specifications. A pattern may contain 
sub-patterns which also define inlets, and so on. A top-level pattern can be seen as a 
class definition, whereas sub-patterns can be seen as (sub-)method definitions. 
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A notification pattern is part of a pattern definition. It specifies the output behavior 
of a pattern in terms of notifications. An object interested in a particular notification of 
a pubhshing object can subscribe to that notification. The subscription information is 
comprised of, amongst others, the name of the notification, the identity of the subscriber 
and the pattern to be invoked in the subscriber. Notifications are issued through an outlet 
by means of a bang-bang (!!) primitive. As a matter of fact, notifications are not only 
used for implementing the Observer design pattern, but they are also used for getting a 
reply value as a result of sending a request to some object. The basic idea is to send a 
message to an object and then wait for a notification to be received in an inlet following 
the send primitive. The concept of notification patterns has been explored in Talktalk 
[ 1 ]. 

The type or types of a pattern are provided by interfaces. A pattern that implements 
an interface has the type of that interface. As in Java, multiple interface inheritance is 
supported in BCOOPL. That is, an interface may extend one or more sub-interfaces. In 
contrast to Java, BCOOPL interfaces contain sequence information. 

2.2.2 Computational Model The computational model of BCOOPL is based on 
message passing and concurrent objects. Objects are instantiated from patterns by ex- 
ecuting the new primitive. A pattern may contain sub-patterns and their corresponding 
objects are instantiated imphcitly whenever a super-pattern is instantiated. A concep- 
tual model of an object and its sub-objects is shown in Figure 3. Each object has an 
unique I.D., which is used as an address for message exchange. Objects communicate 
with other objects by means of a restricted from of asynchronous message passing in the 
sense that the partial ordering of messages sent from one object to another is preserved. 
An object receiving a message does not process the message right away, instead the 
message is placed in an unbounded message buffer. The dispatcher searches the mes- 
sage buffer on afirst-come-first-servedha&is of acceptable messages. The acceptabihty 
of a message is determined by the state of patterns in execution. If an acceptable mes- 
sage is found, the dispatcher passes the message on to the corresponding (sub-)object’s 
inlet, otherwise it waits for the arrival of new messages. Thus, the communications be- 
tween objects can be summarized as synchronous message buffering, but asynchronous 
message processing. 

A Computation in BCOOPL is achieved by sending messages. This includes control 
flow structures like selection (if-then-else) and repetition (while-do). A computation 
proceeds as follows. After a message has been received in an inlet, the sequence of 
primitives following the inlet are executed until one or more inlets (sub-patterns) are 
encountered. Regular expression operators, such as the selection (-t) and the repetition 
(*), imply choices. Each branch resulting from such a choice must be guarded with an 
inlet. That is, the choice to follow a particular branch is made by sending an appropriate 
message. There is no such concept as non-deterministic choices. 

Within an object and its sub-objects the one-at-a-time principle of executing primi- 
tives apphes. Multiple execution threads may occur within a tree of (sub-)objects, which 
are introduced with the interleave operator (||). At most one thread, however, is active 
at any one time. Thread switching occurs at the time the active thread runs into one or 
more sub-patterns. Because almost every computation step is expressed in terms of mes- 
sage passing, thread switches occur frequently, which amounts to a semi-parallel object 
model. The next active thread is selected on the basis of the acceptabihty of the pending 
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Fig. 3. Object model. 



messages in the message buffer. This is a fair scheduHng mechanism since the active 
thread cannot interfere with the scheduling of other threads, unless it claims exphcitly 
the exclusive ownership of the object by means of the synchronize operator (<Cexpr 
In contrast to intra-object concurrency, top-level objects (and their sub-objects) operate 
on a truly concurrent basis. 

2.23 Interface and Pattern Specification An interface is identified by a name and 
may extend one or more base interfaces. It is defined by means of an interface interac- 
tion term. 

interface Fnteiface Name 
extends linteifaces]opt 

defines [ 

interface interaction term 

'\ctpt 

An interface interaction term is specified using the following syntax: 

client specifications i — Pattern Name {input args) ^ {notification pattern) [ 
regular expression over interface interaction terns 

]opt 

An interface interaction term corresponds with a (sub-)pattern definition that imple- 
ments the interface. It defines the pattern name, the formal input arguments, a notifica- 
tion pattern that specifies sequences of notification messages, and chent specifications. 
An interface interaction term is recursively defined as a regular expression over interface 
interaction terms leading to a hierarchical interface specification. The regular expres- 
sion operators used for constructing an interface and their meaning are summarized in 
Table 1. 

Ghent specifications denote the parties that are allowed to invoke the corresponding 
pattern. They are defined by any combination of the foUowing: by interface name, by 
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Expression Operator 


Meaning 


<E> 


synchronize 


E is executed uninterupted 


E||F 


interleave 


E and F may occur interleaved 


E-IF 


selection 


E or F can be selected 


E;F 


sequence 


E is followed by F 


E ^ 


repetition 


Zero or more times E 


E [m, n] 


bounded rep. 


i times E with m <i <n 



Table 1. Semantics of regular expression operators. 



interface name set, or by object reference set The sets are used to dynamically specify 
the clients that are allowed to interact. A pattern implementing such an interface is 
responsible for the contents of a particular set. 

Notifications issued by a pattern are guaranteed to be emitted according to the de- 
fined sequences specified in its notification pattern. A notification pattern is defined as 
a regular expression over notification terms that are specified as follows: 

Notification Name (output args) 

The CO- and contra-vaiiance rules apply for specifying interfaces. An interface in- 
teraction term may be redefined in a derived interface. The types of the input arguments 
must be the same as or generahzed from the argument types of the base interface (i.e., 
contra-variance rule). In contrast, a notification pattern may be extended in a derived 
interface, both in terms of notification output arguments having derived interfaces (i.e., 
co-variance rule) and additional notifications. 

The interface Any acts as a base type for every other interface. That is, every inter- 
face extends Any imphcitly. Any is defined as: 

intferfat^ Any 

As an example of interface specification, consider the interface for a scarce resource. 
A resource must be acquired before it can be used, and after it has been used, it must 
be rehnquished to allow other objects to use it again. The interface for a resource could 
read as follows: 

interface Resource defines [ 

Anyi-^- ()=S()[ 

( 

Any ^ acquire () =S (done()) ; 

Any ^ use (in : SomelnArg) =s (result(out : SomeOutAr^)) * ; 

Any ^ relinquish 0 =S (done(» 

)* 

] 

1 



This interface enforces cycles of acquiring, using, and rehnquishing the resource. 
Patterns and sub-patterns are defined identically. The overall structure of a pattern 
is the following: 

client/sen'er specifications pattern Pattern Name (input args) =)> (notification pattern) 
implements [inteifaces]opt 
declares [local variables]opt 
does [ 

pattern implementation; a regular expression over primitives and sub-patterns 
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Client/server specifications are defined in patterns similar to client specifications 
in interfaces. Client specifications identify the kind of objects that may communicate 
with a pattern, whereas server specifications denote declaratively specified linkages to 
notifications. The syntax for server specifications is as follows. 

Local Variable.Pattern Namei.Pattern Name2-' ■ ■ .Pattern Name n. Notification Nameiformal arguments) 

Object Set.Pattern Namei.Pattern Name2‘‘ ■ ■ .Pattern Namen.Notif cation Nameiformal arguments) 



Notification linkages are estabhshed at mn-time. An assignment to a variable in- 
volved in notification Hnkage results in first aboHshing the current Hnk, provided the 
variable is not nil, then assigning to the variable, and finally establishing a new link if 
the variable does not equal nil In the case of an object set, a notification hnk is estab- 
hshed when an object is added to the set, hkewise it is abohshed when the object is 
removed from the set. 

The behavior of a pattern is defined in the does section. It is defined as a regular 
expression over primitives and sub-patterns. The supported primitives are summarized 
in Table 2. As an example, an implementahon of the Resource is given below. 



Any i-A pattern resource 0 0 implements [ Resource ] does [ 

( 

Any I— pattern, acquire () =>► (done()) does [ ! ! done() ] ; 

Any I— pattern use (in : SomeTnArg) (resuIi(oui : SomeOuiAig;)) 

declares [resuIiAi^ : SomeOuiArg ; ] 
does [ 

// use the resource 
resuIlArg := ■ ■ ■ ; 

! ! resuIi(resuIiAr 2 ) 

Any 1—^ pattern relinquish () (doneO) does [ ! ! done() ] 

] 



23 Augmenting UCMs with BCOOPL Interface Specifications 

The UCM notation has been augmented with an extension and some notational short- 
hands in order to have a better match with BCOOPL ’s language features. The augmen- 
tations are depicted in Figure 4. The extension is the more rigorously defined semantics 
of a scenario in progress along a path within a component. As in BCOOPL, the one-at- 
a-time regime apphes, which means that only one thread of control is active at any one 
time, although multiple threads may be in execution on an interleaved basis. However, 
a scenario in execution can claim exclusive control over a component by means of en- 
closing a path segment in <C and markers. Notational shorthands have been provided 
for interface specifications, (asynchronous) message exchanges, and synchronization. 

3 Running Example 

A simphfied, but reahstic model of a cHent/server system is used as a running example. 
The chent and the server share a resource that can only be used by one party at a time. 
In this example we show that a component, in this particular case the server, operates 
perfectly well in isolation, but it may fail when it is subjected to a composition in 
which another component, in this case the chent, uses the same shared resource. This 
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Primitive 


Abstract Syntax 


Remarks 


Assignment 


variable := FQNexpression 


A FQN (Fully Qualified Name) 


New 


newPattem Name 


denotes an object. It is specified 
as: ( Pseudo- )Variable.Pattem 

Nameic • • .Pattern Namen 
The designated pattern is in- 


Send 


FQN (message args) 


stantiated along with its sub- 
patterns resulting in an object 
tree. Unreferenced objects are 
reclaimed by a garbage collec- 
tor. 

A message is sent to the object 


Request 


request FQN (message args) 


designated with the FQN 
Identical to a send with the ex- 


ception that a reply (i.e., a no- 
tification) is sent only to a sub- 
object of the object that issued 
the request. 

Inlet (Pattern) CS specs pattern Name (in args) A message is received in an in- 


Lightweight 


(notifications) does [ • • ■ ] 
CS specs 1-^ pattern Name (yars) 


let which is implicitly defined 
in a pattern. 

In contrast with an ordinary in- 


Inlet (Pattern) 
Outlet 


let, a lightweight inlet does not 
introduce a local scope. The re- 
ceived message arguments are 
stored in the designated vari- 
ables. 

^Notification Name (message args) A notification is issued. 


Client/server add or remove 


The add and the remove are cur- 


Set Opera- 
tions 
Delegate 


not discussed further 


rently the only supported oper- 
ations. 



Table 2. Primitives. 



is a typical iLLustration of architectural mismatch where components make (possibly 
undocumented) assumptions on the environment [11]. The interface for the server is 
shown below, the interface and the implementation of the resource was already given in 
a previous section. 



interface Server defines [ 

Any I— (resource : Resource) () [ 

Any i-A service (in : SomelnArg) (resuIi(oui : SomeOuiArg)) * 




A server is initiaUzed with a resource. Most likely, the server wiU use this resource, 
although it is not clear when the resource is used. This kind of behavior specification 
cannot be deduced from the interface specification. For this and other reasons not dis- 
cussed here we have proposed a grey-box approach for component specification com- 
prised of UCMs and BCOOPL interface specification [6]. The combination of UCMs 
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Fig. 4. UCM augmentations. 



and BCOOPL interfaces gives us the opportunity to reason about the internal behavior 
of components and their interactions with other components. 

The interactions between the server and the resource is depicted in Figure 5. A 
simphfication is shown in the same figure in which the interactions between the two are 
modeled with responsibhity points and the resource component is removed from the 
graphical representation altogether. 



Serv'er/Resoiirce interactions Simplified Server/Resoiirce interactions 




Fig. 5. Server/Resoufce interactions. 
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When the server is brought together with a cHent that uses the same resource in a 
manner depicted in Figure 6 the system might come to a halt because of deadlock. The 
interface for the cHent is given below. 
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Fig. 6. Client/Server/Resource interactions. 



interface Clieni defines [ 

Any i-A (server : Server^ resource : Resource) =)>()[ 

Any 1-4 doll (in : SomelnArg) =|> (done(oui : SomeOuiArg)) * 




Both the chent and the server try to acquire the resource just after the client has sent 
a service request to the server. The system deadlocks if the chent acquires the resource 
first. In that case, the chent will never receive a reply (in the form of a result notification) 
from the server, since the server wants to acquire the resource but the resource was 
already acquired by the chent and wih be rehnquished only after the server has rephed. 
So, the chent as weU as the server want to proceed, but they cannot do so because they 
are awaiting an event that wiU not happen. In other words, this is a classical example of 
deadlock. 

If, on the other hand, the server acquires the resource first, the system proceeds as 
intended. The server uses and then rehnquishes the resource and continues with issuing 
a result notification. In the meantime, the chent tries to acquire the resource and use 
it once before processing the result notification. Eventuahy, the chent wih succeed in 
doing so. After the chent has processed the reply from the server, it proceeds with 
another use of the resource before rehnquishing it. 

4 Preparing UCM/BCOOPL Interface Specifications for Analysis 

A component composition comprised of UCMs and BCOOPL interface specifications is 
translated into a corresponding BCOOPL program before it is analyzed. The translation 
can be done in an automatic and unambiguous way. We demonstrate how typical UCM 
constructs, hke the AND/OR-fork/join and the request, translate into BCOOPL. The 
translation scheme for the AND-fork/join is given in Figure 7. The basic idea is to 
send two (asynchronous) messages to patterns defined in the object itself. The patterns 
that handle these messages are contained in an interleave (||) expression, so these two 
patterns are executed concurrently. 
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^prc. ! 

seIf.palhAO ; 
self.paihBO : 

( 

self I— pattern paihA () =>► () does [ 



Rpre 


Ra 


Rposl 




Rb 





] 

self i-A pattern paihB () () does [ 

Rb 



); 

Rpost i 



Fig. 7. Translation scheme for the AND-fork/join. 



The translation scheme for the OR-fork/join is along the same hnes as for the AND- 
fork/join as is depicted in Figure 8. The forked paths are embedded in an alternative (+) 
expression rather than an interleave expression. In contrast to the AND-fork/join, only 
one path will actually be executed. The path to be executed is selected on the basis of 
message receipt in an inlet defined at the beginning of a forked path. 



pathA palhB 




( 

Any 1-^ pattern paihA () =S () does [ 
Ra 

1 

+ 

Any i-s- pattern paihB () =s () does [ 

Rb 

] 

>; 

Rpoat ^ 



Fig. 8. Translation scheme for the OR-fork/join. 



As discussed before, a request is spHt into a message send followed by an inlet in 
which the reply notification is received. In between the message send and the receipt of 
a reply in an inlet, an object can perform additional computations that run concurrently 
with the activities of the object that services the request. The translation scheme for the 
request is given in Figure 9. 

Given these translation schemes, we show in Figure 10 how the cfient/server exam- 
ple is translated into BCOOPL. 
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^pra I 

request someObjeci.service() ; 

Rai 

someObjeci.service.resuIi() i— ^ pattern result () ; 

Rpofit ! 



F^.9. Translation scheme for the request. 



5 Analyzing Compositions 

Finally, we show how a generated BCOOPL program is analyzed. This involves the 
following three steps: 

1 . type inferring pattern expressions; 

2. building a communication graph annotated with thread information; 

3. analyzing the composition with temporal orderings. 

The first two steps are used to represent the program in such a way that it can be 
readily used in the final, analysis step. 

The first step is type inferring the generated BCOOPL program in order to iden- 
tify the types of pattern expressions as precisely as possible. Type information is used 
to deduce communication targets. In principle, communication targets can be distilled 
from a UCM. However, to better deal with polymorphism, type inferring a BCOOPL 
program yields more accurate type information. In addition, type information is also 
used to check the program’s sanity. For instance, we can verify whether a variable that 
holds a target for a message send is initi ali zed before it is used. Due to lack of space, 
type inferring BCOOPL programs is not discussed in this paper. A detailed discussion 
of type inference in Talktalk (a predecessor of BCOOPL), which is based on data flow 
equations, can be found in [5]. 

The second step is to build a communication graph detaihng the send-to and 
received-by relations between (sub-)pattems. This graph is akin to a call graph for im- 
perative programs. The communication graph is annotated with thread information to 
show the threads in which message sends and receipts occur. 

In the final third step, the composition is analyzed by constructing directed graphs 
that show the temporal orderings between events (i.e., a message send and a message 
dispatch). Such graphs are constructed for each scenario specified in a UCM. Any vio- 
lation of ordering constraints in a graph can be seen as a problem indicator. 

For analysis purposes, BCOOPL ’s computational model is shghtly adapted in order 
to simplify reasoning about component compositions without actually changing the 
semantics. A single message buffer is used instead of equipping each component with 
its own buffer. In reahty, BCOOPL objects are equipped with individual buffers, which 
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// Clieni implemeniaiion 

Any I— pattern client (client : Client^ resource : Resource) =>► () implements [Client ] does [ 

Any I— pattern dolt (in : SomelnArg) =>► (done(out : SomeOutArg))) 
declares [ resuItArg : SomeOutArg ; ] 
does [ 

servenservice(in) ; 

request resource.acquireO ; resource.acquire.doneO i-A pattern resourceAcquired () ; 

request resource.use(in) ; resource.use.resuIt(out : SomeOutArg) 1-4 pattern resourceResuIt (resuItArg) ; 

request server.service.resuIt(out : SomeOutArg) i-A pattern serverlsDone (resuItAi^) ; 

request resource.use(in) ; resource.usej^suIt(out : SomeOutAig:) 1— pattern resourceResuIt (resuItArg) ; 

request resource.reIinquish() ; resource.reIinquish.done() 1— pattern resourceRelinquished () ; 

!! done(resuItAig:) 




// Server implementation 

Any 1—^ pattern server (resource : Resource) () implements [ Server ] does [ 

Any i-A pattern service (in : SomelnArg) (resuIt(out : SomeOutArg)) 
declares [ resuItArg : SomeOutArg ] 
does [ 

request resource.acquireO ^ resource.acquire.done() 1— pattern resourceAcquired () ; 

request resource.use(in) ; resource.use.resuIt(out : SomeOutAig:) 1-4 pattern resourceResuIt (resuItArg) ; 

request resource.reIinquish() ; resource.reIinquish.done() 1— pattern resourceRelinquished () ; 

!! resuIt(resuItAig:) 




// System configuration (main start-up code) 

Any I— pattern main () () 

declares [ 

client : Client ; 
server : Server ; 
resource : Resource ; 
inArg : SomelnArg ; 
outArs : SomeOutArg ; 

] 

does [ 

client := new client ; 
server := new server ; 
resource := new resource ; 
inArg := new somelnArg ; 

resourceO ; 

client(server» resource) ; 
server(resource) ; 

request cIient.doIt(inArg) ; cIient.doIt.done(out : SomeOutArg) 1— pattern clienllsDone (outArg) 



Fig. 10. Client-Server system translation into BCOOPL. 



is of course a necessity for truly concurrent and distributed computing. The following 
rules hold: 



- The temporal ordering of messages sent by one component is preserved in the mes- 
sage buffer. 

- Message dispatching is based on a first-come-first-served basis of acceptable mes- 
sages as far as an individual object (and its sub-objects) is concerned. Each object 
publishes its acceptable (sub-)patterns (i.e., there corresponding (sub-)objects are 
ready to be executed). A non-deterministic scheduHng poHcy appHes for choosing 
the next object to be executed. 
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As a consequence, the following general rule is valid: tmessags buffering < 
tmessage dispatching (abbreviated as tbuf < tdisp)- In words, a message is buffered be- 
fore it is dispatched. 

There is a one-to-one correspondence between regular expressions and state ma- 
chines. Therefore, interface specifications, which are defined as regular expressions 
over interface interaction terms, imply temporal orderings. A typical example is given 
in the interface specification of a resource, which states that a resource cannot be used 
before it is acquired, and that it cannot be acquired again before it is rehnquished. This 
can be formahzed as shown below. 



simplified Resource interface specification : (acquire; use*; relinquish)* 

^acquire disp^ tuse disp^^ tuse disp^^^^ I'relinquish disp^ l^acquire disp^^^ 

with index i denoting the outer repetition cycle number, and 
with index j denoting the inner repetition (use) cycle number 

The temporal orderings can be shown as a directed graph in which the < relation is 
represented as a directed edge between two nodes. 

We are now in the position to analyze a composition on a scenario basis. Before 
discussing how all scenarios can be enumerated systematically, we show the general 
approach by taking the cHent/server system as an example. In this particular case, two 
scenarios can be recognized. In the first scenario, the chent acquires the resource first. 
The temporal event orderings of this scenario are depicted in a sHghtly simpHfied form 
in Figure 1 1 (for one thing, the invocations of the resource’s use pattern have been 
omitted). As can be seen in the figure, the scenario leads to a situation in which a 
notification (the wsult notification issued by the server) is dispatched before it is issued 
(i.e., buffered). This is a violation of the rule that a message just be buffered before it 
can be dispatched. Therefore, we conclude that the system comes to a halt as a result of 
deadlock. A second scenario can be devised in which the server requires the resource 
first. Due to space fimitations we will not give a graphical representation here. In this 
case, however, no contradictions are found. 

The enumeration of all scenarios is done by systematically considering all paths in a 
UCM. In particular, the OR-fork introduces two independent (sub-)scenarios, whereas 
the AND-fork leads to two concurrently mnning threads. In principle, aU permutations 
of events in concurrent threads yield the set of scenarios to consider, which obviously 
leads to an explosion of scenarios. Fortunately, there are ways to prune the scenario 
set. First of all, we can check whether two threads are independent of each other by 
consulting the communication graph annotated with thread information. If components 
involved in communications originating from one thread do not use the components 
invoked from the other thread, then from an analysis point of view the execution of both 
threads can be regarded as a single scenario. Secondly, the temporal orderings imphed 
by pattern expressions enforce particular message sequences (e.g., the acquire ; use * 
; relinquish message sequence of a resource). This means that certain permutations of 
events in concurrent threads do not apply. For instance, in the chent/server example, a 
resource can only be used if it has been acquired. Therefore, we do not have to consider 
the permutations of aU resource uses stemming from the chent and the server. 
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Fig. 11. Scenario leading to deadlock. 



In conclusion, by analyzing temporal orderings, we might find contradictions which 
are an indication for errors in a specification, such as the presence of deadlock. Thus, 
erroneous behavior is found by considering all possible scenarios. 

6 Discussion 

We have shown how the presence or absence of deadlock in a system comprised of 
components can be proved. Another application area of this approach is to investigate 
reachabihty, a concept closely related to the notions of goals. By analyzing a scenario, 
we can verify whether the intended goals of that scenario are reached or not. This notion 
can be reformulated in terms of events. That is, we can analyze whether a particular 
event (message sending or message dispatching) or a set of events that is associated 
with a certain goal does happen. 

As discussed before, systems comprised of compositions of concurrently run- 
ning components are difficult to analyze due to the state explosions imphed by non- 
deterministic behavior of concurrent components. Techniques have been developed to 
handle large number of states. For instance, a model-theoretic reasoning tool (i.e., a 
model checker), called the Symbohc Model Verifier (SMV), has been used to verify 
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models with more than 10^® states [3]. Although practical, but relatively small systems 
can be verified in this way, real-size systems can easily outdo the aforementioned num- 
ber of states. A scenario-based approach fike UCM reduces the number of possibifities 
to consider. The functionafity provided by a system comprised of components is typi- 
cally not equal to the sum of the functionahties provided by its parts. In general, com- 
ponents will be underutihzed. UCM scenarios are employed to show what functionafity 
of components is actually used by the system as a whole. This reduces the complexity 
involved in analyzing component systems. Furthermore, self-contained subsystems can 
be analyzed in isolation. An abstraction (a simplified model) of the subsystem can then 
be substituted in the overall system. Despite the reductions in complexity, the analysis 
of component compositions remains a computational expensive process. 

It is interesting to explore the extent of what can be analyzed by the combination 
of UCM and BCOOPL interfaces. To put it differently, does the combination specify 
the behavior of a component composition completely? This turns out not to be the 
case. For one thing, pre- and post-conditions cannot be expressed formally in BCOOPL 
interfaces. (However, Pre- and post conditions can be specified informally by means 
of responsibility points in UCMs.) Although temporal orderings implied by regular 
expressions and client specifications can be seen as pre-conditions, there is no way 
to state pre- and post-conditions in terms of state variables. Thus, BCOOPL offers a 
static construct for specifying enabling conditions, rather than taking runtime, dynamic 
behavior into account. The advantage of this approach is that we do have a specifications 
of temporal orderings, which are hard to deduce from pre- and post-conditions alone. 

There are no principal reasons not to support state variables in BCOOPL interfaces. 
It should be noted that the inclusion of state variables is not a violation of the principle 
that an interface should not commit to implementation details. For instance, size is 
an intrinsic property of a stack. Pre- and post-conditions for stack operations can be 
specified in terms of size without committing to an implementation yet. This notion is 
central in the component specification method Catalysis [9]. Catalysis also includes the 
concept of refinement. Interface (type) specifications can be gradually refined into an 
implementation as long as it is assured that an implementation is in conformance with 
its type. 

Nevertheless, potential problems can be pinpointed by the combination of UCMs 
and BCOOPL interfaces. In some cases, however, more problems wifi be spotted than 
can actually happen at runtime due to the fact that enabling conditions cannot be spec- 
ified precisely enough at present. The use of state variables in interface specifications 
wifi be explored in the near future. 



7 Concluding Remarks 

We have discussed an approach for modeling component compositions and analyzing 
them statically. UCMs and BCOOPL interfaces strike the balance between understand- 
abifity (by humans) and preciseness (for reasoning about component compositions). 
Interacting UCM scenarios give a high level, easy to understand overview of the be- 
havioral aspects of a system. UCM scenarios in combination with BCOOPL interface 
specifications allow to reason about it formally. In addition, the use of UCM scenarios 
helps in reducing the complexity of the analysis process. 
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Future work include modeling gradual refinements (from type specification to im- 
plementation) and investigating quality attributes (e.g., performance) other than behav- 
ior that are amenable for analysis purposes using the approach discussed in this paper. 

In addition, we want to look at tool support for analyzing real-size systems. 
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Abstract. Product instantiation is one of the less frequently studied activities in 
the domain of software product lines. In this paper, we present the results of a 
case study at Axis Communication AB on product instantiation in an industrial 
product line, i.e. five problems and three issues. The problems are concerned 
the insufficiency of functional commonality, features spanning multiple 
components, the exclusion of unwanted features, the evolution of product line 
components and the handling of initialization code. The issues discuss 
architectural compliance versus product instantiation effort, quick-fixes versus 
properly engineered extensions and component instantiation support versus 
product instantiation effort. The identified problems and issues are based on the 
case study, but have been generalized to apply to a wider context. 



1. Introduction 

The development and evolution of software is expensive and complex, causing few 
software development and maintenance projects to deliver on schedule, within budget 
and at the required quality level. For decades, the reuse of existing software has been 
proposed as the most promising approach to attack these problems. In the earliest 
proposals, software components represented functions that could be reused [11], 
whereas later proposals suggested object-oriented programming, i.e. the use of classes 
and inheritance, to achieve reuse of software [8]. During the second half of the 1980s, 
it was identified that for software reuse to be beneficial, the components need to be of 
larger granularity than functions and classes, which lead to the introduction of object- 
oriented frameworks [8,6] and, later, component-oriented programming [14]. 

All the approaches discussed above aimed at achieving community-wide reuse of 
software assets. Although successful attempts exist, in general one has to conclude 
that the promised benefits of software reuse have not been achieved. One explanation 
is that the software reuse community tried to overcome two dimensions of complexity 
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at once, i.e. reuse between software products and reuse over organizational 
boundaries. In response to this, the notion of software product lines, aiming at reuse 
between software products, but within an organization has achieved considerable 
amounts of attention during recent years, e.g. [9, 7, 16, 4]. 

In this paper we focus on the instantiation of products within a software product line. 
In particular, we present the problems and issues that we identified in a case study 
that we performed at Axis Communications AB. Since product line based 
development differs significantly from the traditional approach to developing 
software, it is reasonable to assume that new problems or new variants of known 
problems exist. Although we earlier have published case studies discussing software 
product lines in general [3], product line components [2] and product evolution [15], 
to the best of our knowledge, few research results are available that address product 
instantiation in software product lines. 

The primary goal of our case study was to identify the problems and issues associated 
with the instantiation of software products in software product lines. The reasons for 
choosing Axis Communications for the case study are that we have had cooperation 
with this company for several years, it has a few well-established software product 
lines that have been operational for several years and the company employs a 
philosophy of openness. Finally, we believe that the company is representative for 
‘normal’ software development organizations, e.g. development departments of 10 to 
100 engineers and developing products sold to industry or consumers. 

The most appropriate method to achieve our goals with this case study, we concluded, 
was through detailed analysis of a number of software products, discussions with 
system architects and studying component and product documentation. For practical 
reasons, we chose to narrow the scope of the study to products within the same 
product family. Although the analyzed products belong to the same product family, 
the variety of configurations concerning both software and hardware gives rise to 
many interesting problems that can be found in other product families as well. 

The contribution of the paper, we believe, is that it identifies a set of important 
problems and issues associated with the instantiation of products within the context of 
software product lines. Awareness of these problems and issues allows practitioners 
to minimize the negative effects and researchers may use these topics in their 
research. 

The remainder of this paper is organized as follows. In the next section, we present 
the case study company and the software product line that is the focus of our case 
study. Section 3 discusses the process of product instantiation within the product line. 
In section 4, the problems that we identified are discussed, whereas section 5 
discusses the issues that we recognized. Related work and our conclusions are 
presented in section 6. 



2. Axis Communication AB 

Axis Communication was incorporated in 1984 and developed a printer server 
product that allowed IBM mainframes to print on non-IBM printers. The first product 
was a major success that established the base of the company. In 1987, the company 
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developed the first version of its proprietary RISC CPU that allowed for better 
performance and cost-efficiency than standard processors for their data- 
communication oriented products. Today, the company develops and introduces new 
products on a regular basis. Since the beginning of the ‘90s, object-oriented 
frameworks were introduced into the company, and since then, a base of reusable 
assets is maintained, from which most products are developed. 

Axis develops IBM-specific and general printer servers, CD-ROM and storage 
servers, network cameras and scanner servers. Especially the latter three product 
types, i.e. the network devices, are developed and evolved using a common product 
line architecture and reusable components. In figure 1, an overview of the product- 
line and the product architectures is shown. The organization is more complicated 
than the standard case, with one product-line architecture (PLA) and several products 
below this product-line. In the Axis case, the software product line is hierarchically 
organized, i.e. there is a top product-line architecture and the product-group 
architectures, e.g. the storage-server architecture. Below these, there are product 
architectures, but since generally several product variations exist, each variation has 
its own adapted product architecture, because of which the product architecture could 
be called a product line architecture. 




Figure 1. The Axis Software Product Line. 

Orthogonal to the products. Axis maintains a product-line architecture and a set of 
reusable assets that are used for product construction. The main assets are a 
framework providing file-system functionality and a framework providing a common 
interface to a considerable set of network protocols, but also smaller frameworks are 
used such as a data chunk framework, a smart pointer framework, a ‘toolkit’ 
framework providing domain-independent classes and a kernel system for the 
proprietary processor providing, among others, memory management and a job 
scheduler. In figure 2, the organization of the main frameworks and a simplified 
representation of the product-line architecture is shown. 
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Figure 2. Overview of the Main Frameworks Used in Axis’ Products. 



3. Product Instantiation 

In this section, we describe the software architecture and the process of product 
instantiation for the products in the network device product line. We focus on the 
StorPointCDElOO product, which is a caching CD-ROM server. It is a member of the 
storage product family, which consists of harddisk, cd-rom, jaz-drive servers. The 
process of product instantiation consists of three steps, i.e. task scheduler 
configuration, service configuration and component parameterization. Finally, we 
discuss the variability mechanisms used in the software product line. 

The Architecture 

The underlying hardware platform is a proprietary system-on-a-chip solution. The 
ETRAX 100 chip is a highly specialized and optimized chip designed to boost 
network performance. It provides built-in SCSI/IDE ATA-2 and lO/lOOMbit ethemet 
interface. It is also equipped with a flash memory that allows remote updating of the 
software system. Like most embedded systems, the software runs on top of a micro 
kernel, in this particular product the OSYS real-time kernel. OSYS is an in-house 
developed kernel extended with a task-scheduler and a mail-communication package. 

Product Template 

Axis maintains a generic architecture template from which individual products are 
derived. The template is a skeleton of integration code connecting the kernel, the 
mailbox-communication package, the job scheduler, the frameworks and the 
components. The high degree of similarity between the product architectures allows 
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all product configurations to be combined in one single code base. The product 
specific code is attached to the reference-product through a C-function interface, 
thereby allowing the use of product- specific, but type-equivalent components. The 
‘completeness’ of the reference product results in that many product-specific variation 
points have become distributed over the reference product, cross-cutting the 
architecture. In order to support all configuration subsets, preprocessor directives are 
used as a mechanism to attach respectively detach components at compilation. This is 
performed through a special header file, specific for each product configuration. 

An overview of the architecture, the interaction of the mailbox-communication, tasks 
and the jobs is presented graphically in figure 3. 




Figure 3. A Simplified View of the Core Product Architecture. 



Since the reference product specifies the component connections, the remaining 
configuration activities include the definition of the task-list for the scheduler and the 
parameterization of the mailbox handler. 

Task-Scheduler Configuration. The task scheduler is configured via a globally 
accessible C-struct. Each product variant supplies its own configuration by explicitly 
linking it into the final product. Each entry (task) is a function pointer referring to a 
startup routine in a ‘active’ component (typically a framework loop), which is 
scheduled using a round-robin algorithm. 

struct tasks [ ] = { 
idle_main, 
j ob_scheduler , 
f lashglow, 
protocol_main 

}; 

The ‘flashglow’ handles run-time updates of the executable-binary, and the 
‘protocol_main’ handles incoming network requests (jobs) and inserts them into the 
queue of the ‘job_scheduler’. 

Service Configuration. Services are typically component plug-ins of a framework. 
Inside each component resides a start-up routine used for initialization and 
registration in the framework. When the system boots up, the scheduler posts 
‘public_create’ and ‘public_init’ messages via the mailbox. The mailbox itself 
contains hard-coded function calls referring to each start-up routine for all available 
components. When the messages arrive, first the frameworks are initialized. 
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Subsequently, the components are initialized and registers themselves at the 
frameworks. The initialization via the mailbox and component start-up routines 
provides a uniform way to add and remove services. At compilation a specific 
configuration can choose whether or not to include a service. From the reference 
product’s perspective the components and frameworks are detached from each other, 
hut in reality implicit dependencies exist. This is discussed in the next section. 

Below, a small code fragment is shown illustrating the use of precompiler directives 
in the mailbox. 

MailBox: : async_message (<init message>) { 
switch (<message>) { 

#if SMB_INCLUDED 
case SMB_PROG: 

smb_main ( <init> ) ; 
break; 

#endif 

#if APPLETALK_INCLUDED 
case APPLETALK_PROG : 

appletalk_main ( <init> ) ; 
break; 

#endif 

. . etc . . 

} 

} 

Component Parameterization. Although the frameworks and the framework plug- 
ins seem independent, in reality there is a hard coupling between the components and 
the frameworks. The components are not reused at binary level, but at source code 
level. The source code is a mixture of common code and product-specific code 
guarded by precompiler directives. Consequently, each product variant needs to 
specify preprocessor directives that include respectively exclude pieces of code before 
the product source is compiled into a binary. Both the reference product and the 
components need to recompiled, when a service is added. Enabling or disabling a 
feature frequently requires recompilation of the complete source code tree, since 
features often have relations to multiple components. 

Below, a code fragment from the file system component is shown in which the 
ISO9660 standard implementation is conditionally included, depending on the 
F_ISO9660 and FS_9660 precompiler flags. 

void CDMediumFSPrototypeMap : : init ( ) { 

#if defined(F_ISO9660) && FS_9660 

this- >registerPrototype ( new IS09660Volume ( ) ); 

//Registers filesystem in the factory 

#endif 

. . . etc . . . 



} 
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Summarizing Discussion 

Axis’ product line consists of frameworks, framework plug-ins (components) and a 
configurable skeleton of integration code. Inheritance and C++ templates are used as 
means to obtain variability of the core assets. The architecture is characterized by its 
pervasive use of design patterns, among others Abstract factory. Composite, Observer 
and Singleton. Axis has taken reuse beyond the availability of core assets by 
providing a skeleton of generic integration code that, once configured, represents a 
complete, functional product. This approach has allowed for taking advantage of the 
similarities within the product family to its full extent. 

The products in the product family are functionally similar but technically dissimilar. 
The hardware differences between products complicate the successful abstraction of 
common features. Nevertheless, through the extensive use of preprocessing. Axis has 
managed to do it in practice. Typical types of usage are: definition of constants and 
parameters, configuration of orthogonal features that span over multiple components, 
configuration of low-level details hidden behind several abstraction levels, removing 
calls/references/dependencies between assets and enabling/disabling hard-coded 
sections of source code. The downside of the focus on the current requirements is that 
the product line assets are hard to evolve, with consequent effects: increased 
complexity of the source code, non-trivial impact analysis, decreased reuse in testing 
and frequent recompilation of the complete source tree. The solution does have 
several advantages: it maximizes reuse, much more assets can be made “generic”, 
assets can be large without loosing configurability and the size of the executable can 
be kept small since unnecessary features can be left out. In the next section we have 
abstracted some of the problems that we found in our case study into more general 
problems specific to product instantiation in software lines. 



4. Problems 

Based on the analysis of the products in the software product line, interviews with 
software architects and engineers and documentation collected at the organizations, 
we have identified a number of problems associated with product instantiation in 
software product line based development that we believe have relevance in a wider 
context than just Axis Communication AB. Some problems are applicable for 
product-line development in general, while other problems are specific the embedded 
systems domain. 

The problems presented here share two main underlying causes, i.e. variability and 
evolution. The variability required from the reusable assets to meet the needs from 
products in the family creates difficulties because separating general and product 
specific functionality is generally not trivial. In addition, evolution of the products 
and the reusable components that define most of their functionality causes solutions 
and design decisions that were correct at the time they were made to become invalid. 
The problem is that removing the effects of these solutions and design decisions may 
be prohibitively expensive. 
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Functional Commonality Is not Enough 

Problem. The reusable components represent the core of software product lines since 
these capture the commonalities between the products. The more commonalties that 
can be extracted from the products and captured into reusable components, the 
smaller the amount of code that has to be maintained. In addition, larger components 
can be reused more efficient [Szyperski 97]. 

The problem is that commonalities between products cannot always be captured in 
generic reusable components, or at least not without considerable effort. A number of 
instances of this problem exist. First, products may have conflicting quality 
requirements, which require component implementations that are incompatible with 
each other. Second, the products may operate on different hardware platforms and it 
may prove difficult to avoid the effects of these differences in the component. Finally, 
although all or most products in the family may require a particular domain of 
functionality represented by a component, different products or subsets of products 
may require features that cross-cut the component functionality. Since these features 
need to be optional, including or excluding a feature may have effects at several 
locations in the component. 

Example. The product line of Axis Communications comprises a variety of different 
products. The products do not share a common hardware platform. They differ in 
configuration of the hardware chip, among the differences are network connectivity 
hardware, cache, flash memory and bus architecture (SCSI/ATA-2). The products can 
however also differ in peripheral configuration, e.g. data media devices (hd/cd/jaz). A 
product is often released in several configurations focussed on particular market 
segments. Variants aimed for the high-end market are equipped with expensive 
hardware peripherals, whereas low-budget variants often lack these. For example. 
Axis StorPointCDElOO is a caching 100 Mbit cd-rom server while StorPointCDlO is 
a 10 Mbit non-caching cd-rom server. The two products are functionally almost 
identical, except for the difference in hardware and the presence of a cache. The 
commonalties between the product variants do not allow for the development of large 
reusable assets, because the implementation of the additional features of the high-end 
product cross-cuts the component functionality and cannot be handled easily by 
traditional variability mechanisms. 

Causes. One may argue that this problem is caused by a poor decomposition of the 
system into its main components, i.e. a badly designed software architecture. We 
investigated this and found no evidence of this being the case. Instead, we became 
convinced that even an otherwise optimal decomposition may cause such problems. 
The system does simply not allow for a decomposition that supports the variability 
requirements for all aspects, e.g. hardware, features, and domain. As we identified in 
[2], functionality related to the different aspects becomes intertwined early in the 
development process and varying one of these aspects independent of the others once 
developed code exists is very difficult. 
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Single Feature Spans over Multiple Components 

Problem. In the previous section, the problem was that multiple features affected a 
single component. However, the opposite situation, i.e. an optional feature affecting 
multiple components, creates difficulties as well. Configuring a single component is 
manageable, but some features require changes in the interaction between 
components, affecting the software architecture and consequently, the initialization of 
the components. In addition, an orthogonal feature may affect the implementation and 
the variability mechanism of individual components. Features that span over multiple 
components may require architectural variability to the extent that the benefits 
associated with software product lines are compromised. 

Example. As mentioned earlier, the StorPointCDElOO product is a caching cd-rom 
server whereas the StorPointCDlOO product is a non-caching cd-rom server. The 
collaborating components behave and interact differently depending on the presence 
of a cache. If a cache is present, a number of configurational issues needs to be dealt 
with. First, the system must be configured for handling a harddisk (which caches the 
content on the CDs). Second, the harddisk requires a read-write file system to be 
added to the system, in addition to the ISO9660 read-only file system used for the 
CD-ROMs. Third, the cache module needs to be configured (size of cache, block size, 
prefetching, etc.). Finally, the cache must be embedded in the software architecture of 
the product, i.e. the new components have to be integrated with the existing 
architecture and the connections of the existing components need to be reconfigured 
to take benefit from the presence of a cache. The presence of a cache affects the 
parameterization and interaction of several components, and hence sections of several 
initialization routines must be maintained to support a single feature. 

Causes. One can identify two main causes for this problem. First, the reusable assets 
of a software product line are not optimized for a specific product, but for a family of 
products. This means that despite the difficulty of integrating a particular feature in 
the product software, the current decomposition and implementation of the software 
product line may still represent the optimal solution. The second cause is that loose 
coupling is usually favored as a means to decrease dependencies between components 
and achieve reuse on a higher level, but not all features can effectively be handled late 
in the configuration process. 

Excluding Component Features 

Problem. In our experience, components in software product lines are typically 
relatively large entities, e.g. 100 KLOC. Several of the companies that we cooperate 
with make use of object-oriented frameworks to implement the component 
functionality. Several of these frameworks, typically black-box, implement the 
superset of the functionality required by the products in the family. However, since 
the features implemented in the framework typically have relations to other parts 
within the framework and to other frameworks, it is normally hard to exclude 
functionality. Especially in the domain of embedded systems, this conflicts directly 
with, among others, the resource efficiency quality requirement. Variability can, in 
several cases, not be supported at the component level without including unused 




156 Jan Bosch and Mattias Hogstrom 



features. Large reusable components have a complex internal architecture with 
dependencies between its parts, which complicate the exclusion of parts of the 
component. 

An additional complicating factor is the fact that components, during evolution, often 
have a tendency to incorporate implicit dependencies to other components it uses, see 
also [Bosch 99b]. These implicit dependencies make it even harder to exclude unused 
functionality from components. 

Example. One of the components in the software product line studied in this paper is 
a network protocol framework that is reused by all products. It supports a plethora of 
different protocols. Most network protocols are implemented on top of lower level 
protocols. The protocols implement different abstraction levels, ranging from details 
about bit transmission to high-level application logic. For, among others, performance 
reasons, the layers in a protocol stack are rather tightly coupled, which make it hard to 
exclude unused protocols. The tight coupling results in relatively large components. 
The problem is that it generally is very hard to exclude code of a protocol due to the 
hard coupling in the implementation, whereas especially in embedded systems it is 
important to keep the system ‘footprint’ as small as possible. 

Causes. The main cause for this problem is evolution. Often, the first version of a 
component supports the variability requirements on the product line at the time of 
design. During evolution, typically two types of change scenarios cause the discussed 
problem. First, functionality that up to now has been included in all products is 
required to become optional because some new product or a new version of some 
product demands this. Second, new features are added to the component because a 
relevant subset of the product family requires this. During implementation, it is 
discovered that the feature affect the component in multiple locations, complicating 
the implementation of the optionality of the new feature. A final cause is discussed in 
the next section. 

Managing Evolution of Product Line Components 

Problem. The core components represent common functionality that is shared by the 
products. For each product, these components are instantiated and configured to 
support the product specific requirements. The number of contexts in which a 
component is used may be substantial. During evolution of a component, it may 
therefore be difficult to verify compatibility of the new version of the component with 
all contexts in which the component may be used. This results in the situation, 
especially during development, where products that successfully compiled and ran 
one day, suddenly exhibit failures on the next. The problem is a software 
configuration management problem, i.e. how to handle impact analysis on a practical 
level. Too much analysis slows down production and delays the introduction of new 
features in products, whereas too little analysis may require additional work to correct 
new versions of components that caused errors in products that incorporate it. 

Example. Axis Communications has a product release schedule that updates the 
product hardware once or twice per year and the product software three to four times 
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per year. Customers that have obtained Axis’ products can download new versions of 
the software and install it in their product, typically overwriting a flash memory. Axis 
constantly enhances its hardware to support more and more functionality. Initially, a 
SCSI interface was the only interface provided for connecting harddisks and cd-roms. 
But, the low budget market demanded the ATA interface, since the devices are 
considerably less expensive than equivalent SCSI devices. There are fundamental 
differences between the ATA and SCSI interfaces. ATA is synchronous and non- 
multitasking, whereas SCSI is asynchronous and multitasking. Initially, the software 
architecture and the components were designed for SCSI, but later versions of these 
assets had to be adapted to support the ATA interface standard as well, though not 
necessarily at the same time. It turned out to be challenging to incorporate 
modifications in core components that are used in all product configurations, 
especially when the components go through extensive product-specific configuration 
or are used in several contexts, i.e. the component should be able to cooperate with 
many different components. Maintenance turned out to be complex, tedious and time 
consuming, since it was hard to do a thorough up-front impact analysis that covers all 
contexts in all configurations. Incidental dependencies might generate overlap in 
functionality or other conflicts. Occasionally, implemented modifications had to be 
removed and reimplemented differently to solve configuration conflicts. 

Causes. The first cause is that the core components are shared entities and all 
products that incorporate these components are affected by modifications. Whenever 
the core assets evolve, all products evolve simultaneously. Common features get 
ideally inserted in the core components, and hence it is desirable that a change is 
compatible with all affected products. The second cause is the high frequency of 
incorporating modifications in these core assets. This makes it practically impossible 
to verify compatibility for each single modification. 

Initialisation Code Is Scattered and Hidden 

Problem. Although the reusable components are shared by all or most products, the 
components are instantiated and configured for each context in which they are used. 
One problem that we have identified is the location of initialization code. A reusable 
component typically contains two types of initialization code. First, generic 
initialization code that instantiates the internal entities and binds them appropriately. 
Second, context specific initialization code that configures the product-specific 
aspects of the component instantiation. Typically, even generic component 
functionality needs to be configured for product specific details. The ambition of 
software product lines is that products in the same product family to share as much 
common code as possible, but at the same time are not forced to duplicate similar 
code for all variants. The problem is that, due to this structure, initialization code is 
spread out throughout the system. First, this causes potential ordering problems 
during system start-up. Because the code is spread out through the system, it is hard to 
predict whether all components will have all required information, e.g. references to 
other components, available at the time of initialization. Second, it is hard to 
determine whether the initialization code is complete. If it is not, uninitialized 
references and similar problems may cause unpredictable system behavior. 
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Example. The products in the Axis product line are built around a micro-kernel, a 
mailbox and a task handler. These parts are relatively tight coupled with the product, 
since those parts manage most of the initialization for all products. Initialization can 
be viewed as part of the reusable components, since it is shared between all products. 
The problem is that the initialization code in a layered architecture of components is 
usually hard to access and modify. As a consequence, the product specific details are 
actually configured by code from inside the components. One disadvantage is that it 
limits the ways in which a product can be tailored and new products can be 
incorporated, and it is not obvious what code actually is used in a specific product or 
when the code is obsolete and can be removed. 

Causes. The primary cause for this problem is there exist two forces that affect the 
location of initialization code. On the one hand, one can argue that initialization code 
ought to be associated with the instantiated component because it is a logical part of 
it. For instance, classes in object-oriented languages have their own constructor 
method. On the other hand, locating the initialization code for all components in a 
single component allows for more control over the ordering of instantiation and 
binding of components. In addition, product-specific functionality and features that 
affect multiple components can be implemented and instantiated more easily. The 
optimal balance between these forces is, in our opinion, specific to each software 
product line. However, typically, generic component initialization code should be 
associated with the component and most product-specific initialization code should be 
centralized. This consequently leads to initialization code being spread out through 
the system. 



5. Issues 

The problems discussed in the previous section present issues that are plain 
problematic and need to be addressed in software product lines. In this section, we 
discuss three issues. Different from problems, issues discuss the balancing between 
two conflicting forces. 

Architectural Compliance versus Product Instantiation Effort 

One of the primary advantages of software product lines is the fact that newly 
incorporated features automatically become available to all products in the product 
line. Since the cost of incorporating a new feature is shared among several products, 
the maintenance cost per product can be decreased drastically. 

Software products are developed based on the product line assets by deriving a 
product architecture from the product line architecture and, subsequently, selecting, 
instantiating and configuring product line components. When the product 
requirements force the product architecture to deviate considerably from the product 
line architecture, fewer product line components can be used for the product and, 
consequently, more product specific components need to be developed. However, this 
may still be the most cost-effective development strategy because achieving the 
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product requirements by adding product-specific code to product line components 
may require even more effort. 

However, if product line members are less compliant to the product line architecture, 
the advantages associated with product line evolution are diminished because new 
features can only partially be implemented at the product line level and need to be 
implemented per product for the remaining part. 

Thus, a conflict exists between the ease with which a product can be instantiated and 
the cost of evolution in the software product line. The software architecture team for 
the product line has to decide where product architectures may deviate from the 
product line architecture and where the product line concepts need to be enforced. 

Determining Value of an Investment 

New requirements on the software product line assets that have architectural impact 
can generally be implemented in two ways, i.e. using a quick-fix avoiding architecture 
reorganization or by a properly engineered, but expensive implementation. It is 
generally difficult to decide whether to accept the drawbacks of a quick-fix or invest 
in a proper but expensive upgrade. The software product line approach is an 
architecture-centric approach. That means that common features are implemented 
once, but may be reused by any product in the product family. This means that the 
cost of a feature can be divided over multiple products. Unfortunately, architectural 
changes may cause ripple effects onto existing products, forcing these product to 
evolve with the architecture. Proper implementation of a requirement with 
architectural impact is potentially very expensive. 

A quick-fix avoids architecture reorganization and is consequently less effort 
demanding, but degrades the structure and conceptual integrity of the software 
product line. Multiple quick-fixes will cause architecture erosion and decrease the 
economical value of product line assets. 

In order to make a well-founded decision, an investment analysis of the different 
alternatives may be necessary. A large up-front investment may prove useless due to 
unexpected technical issues or market events. A quick-fix requires less effort and 
allows for short time-to-market, but are harmful for maintainability of the product 
line. Since typically no investment analysis is done, quick-fixes are often not replaced 
after the deadline, for instance because few understand the actual implementation and 
because it requires effort that otherwise could be spent on adding new features or 
products to the product line. 

The question is which approach provides the highest return on investment. It is easy 
to criticize quick-fixes, but in some cases it is the preferable approach. The lack of 
economical models make it hard to show the long-term benefits of a software product 
line and how it is affected by various maintenance alternatives. 

Component Instantiation Support versus Product Instantiation Effort 

Components in a software product line are generally large and relatively complex. 
Instantiating and configuring a component for a new member of the product line (or a 
new version of an existing member) may therefore be difficult and time consuming. 
Since this has to be repeated for all product line components used in the product, the 
effort required for the instantiation of the product may be substantial. 
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Although a software component may just provide the source code and the 
documented provided, required and configuration interface, the developers of the 
component are able to provide more advanced instantiation and configuration support. 
For instance, in [13] the use of visual programming tools and domain specific 
languages is discussed to instantiate and configure object-oriented frameworks. The 
availability of such tools can decrease the instantiation effort considerably. 

Although tools supporting component instantiation provide important support for 
product instantiation, there are two important issues that should be considered. First, 
the development of component instantiation support is effort demanding and increases 
the total component development cost substantially. Second, since more software is 
associated with a component, i.e. the component code and the instantiation support, 
the incorporation of new features in the component will become more expensive 
because the instantiation support must evolve as well. 

The balance between providing component instantiation support and product 
instantiation effort is influenced by the number of new features that need to be 
incorporated and by the number of product instantiations that incorporate the product. 
Consequently, component instantiation support is more suitable for stable domains 
where relatively few new features are incorporated. 



6. Conclusions 

The notion of software product lines currently provides the most promising approach 
to increasing reuse of software and, consequently, decreased development cost and 
time-to-market. Several authors have discussed software product lines. For instance, 
in [7], the authors discuss an approach to software product lines consisting of 
application family engineering, component engineering and application engineering 
phases. Although product instantiation is discussed, the problems and issues discussed 
in this paper are not addressed. [16] discuss the FAST method, which is a systematic 
approach creating a software product line with a primary focus on the use of domain- 
specific languages and code generators for these languages. Other work in the area of 
software product lines includes [1, 5, 10]. However, none of these publications 
discusses product instantiation in software product lines in detail. In [4], we discuss 
the development, deployment and evolution phases of software product lines, but do 
not discuss the problems and issues presented in this paper. 

In this article, we have presented the results of a case study identifying the problems 
and issues associated with product instantiation. The case study was performed at 
Axis Communications AB and focussed on their set of CD-ROM products in the 
networked devices product line. We identified five problems and three issues. The 
problems we discussed are concerned with the insufficiency of functional 
commonality, features spanning multiple components, the exclusion of unwanted 
features, the evolution of product line components and the handling of initialization 
code. The issues discuss architectural compliance versus product instantiation effort, 
quick-fixes versus properly engineered extensions and component instantiation 
support versus product instantiation effort. 
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The contribution of the paper, we believe, is that it identifies a set of important 
problems and issues associated with the instantiation of products within the context of 
software product lines. Awareness of these problems and issues allows practitioners 
to minimize the negative effects and researchers may use these topics in their 
research. 
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Abstract. Combinations of C++ features, like inheritance, templates, and 
class nesting, allow for the expression of powerful component patterns. In 
particular, research has demonstrated that, using C++ mixin classes, one can 
express layered component-based designs concisely with efficient implemen- 
tations. In this paper, we discuss pragmatic issues related to component-based 
programming using C++ mixins. We explain surprising interactions of C++ 
features and policies that sometimes complicate mixin implementations, 
while other times enable additional functionality without extra effort. 

1 Introduction 

Large software artifacts are arguably among the most complex products of human 
intellect. The complexity of software has led to implementation methodologies that 
divide a problem into manageable parts and compose the parts to form the final 
product. Several research efforts have argued that C++ templates (a powerful 
parameterization mechanism) can be used to perform this division elegantly. 

In particular, the work of VanHilst and Notkin [29][30][31] showed how one can 
implement collaboration-based (or role-based) designs using a certain templatized 
class pattern, known as a mixin class (or just mixin). Compared to other techniques 
(e.g., a straightforward use of application frameworks [17]) the VanHilst and Notkin 
method yields less redundancy and reusable components that reflect the structure of 
the design. At the same time, unnecessary dynamic binding can be eliminated, 
resulting into more efficient implementations. Unfortunately, this method resulted 
in very complex parameterizations, causing its inventors to question its scalability. 

The mixin layers technique was invented to address these concerns. Mixin layers are 
mixin classes nested in a pattern such that the parameter (superclass) of the outer 
mixin determines the parameters (superclasses) of inner mixins. In previous work 
[4][24][25], we showed how mixin layers solve the scalability problems of the Van- 
Hilst and Notkin method and result into elegant implementations of collaboration- 
based designs. 
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This paper discusses practical issues related to mixin-based programming. We adopt 
a viewpoint oriented towards C++ implementations, but our discussion is not geared 
towards the C++ expert. Instead, we aim to document common problems and solu- 
tions in C++ mixin writing for the casual programmer. Additionally, we highlight 
issues that pertain to language design in general (e.g., to Java parameterization or to 
the design of future languages). Most of the issues clearly arise from the interaction 
of C++ features with the constructs under study. The discussion mainly stems from 
actual experience with C++ mixin-based implementations but a few points are a 
result of close examination of the C++ standard, since they refer to features that no 
compiler we have encountered implements. Even though we present an introduction 
to mixins, mixin layers, and their uses, the primary purpose of this paper is not to 
convince readers of the value of these constructs. (The reader should consult [4], 
[24], [25], [26], or [29] for that.) 

We believe that the information presented here represents a valuable step towards 
moving some powerful programming techniques into the mainstream. We found that 
the mixin programming style is quite practical, as long as one is aware of the possible 
interactions with C++ idiosyncrasies. As C++ compilers move closer to sophisticated 
template support (e.g., some compilers already support separate template compila- 
tion) the utility of such techniques will increase rapidly. 

2 Background (Mixins and Mixin Layers) 

The term mixin class (or just mixin) has been overloaded in several occasions. Mixins 
were originally explored in the Lisp language with object systems like Flavors [20] 
and CLOS [18]. In these systems, mixins are an idiom for specifying a class and 
allowing its superclass to be determined by linearization of multiple inheritance. In 
C++, the term has been used to describe classes in a particular (multiple) inheritance 
arrangement: as superclasses of a single class that themselves have a common virtual 
base class (see [28], p.402). (This is not the meaning that we will use in this paper.) 
Both of these mechanisms are approximations of a general concept described by 
Bracha and Cook [6]. The idea is simple: we would like to specify an extension with- 
out pre-determining what exactly it can extend. This is equivalent to specifying a 
subclass while leaving its superclass as a parameter to be determined later. The bene- 
fit is that a single class can be used to express an incremental extension, valid for a 
variety of classes. 

Mixins can be implemented using parameterized inheritance. The superclass of a 
class is left as a parameter to be specified at instantiation time. In C++ we can write 
this as: 

template <class Super> 
class Mixin ; public Super { 

... /* mixin body */ 

}; 



To give an example, consider a mixin implementing operation counting for a graph. 
Operation counting means keeping track of how many nodes and edges have been 
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visited during the execution of a graph algorithm. (This simple example is one of 
the non-algorithmic refinements to algorithm functionality discussed in [33]). The 
mixin could have the form: 

template <class Graph> 
class Counting: public Graph { 

int nodes_visited, edges_visited; 
public : 

CountingO : nodes_visited ( 0 ) , edges_visited ( 0 ) , GraphO { } 
node succ_node (node v) { 
nodes_visited++ ; 
return Graph: : succ_node (v) ; 

} 

edge succ_edge (edge e) { 
edges_visited++ ; 
return Graph: : succ_edge (e) ; 

} 



By expressing operation counting as a mixin we ensure that it is applicable to many 
classes that have the same interface (i.e., many different kinds of graphs). We can 
have, for instance, two different compositions: 

Counting< Ugraph > counted_ugraph; 
and 

Counting< Dgraph > counted_dgraph; 
for undirected and directed graphs. (We omit parameters to the graph classes for 
simplicity.) Note that the behavior of the composition is exactly what one would 
expect: any methods not affecting the counting process are exported (inherited from 
the graph classes). The methods that do need to increase the counts are “wrapped” 
in the mixin. 

VanHilst and Nothin demonstrated that mixins are beneficial for a general class of 
object-oriented designs [29]. They used a mixin-based approach to implement col- 
laboration-based (a.k.a. role-based) designs [5][15][16][21][29]. These designs are 
based on the view that objects are composed of different roles that they play in their 
interaction with other objects. The fundamental unit of functionality is a protocol 
for this interaction, called a collaboration. The mixin-based approach of VanHilst 
and Nothin results in efficient implementations of role -based designs with no redun- 
dancy. Sometimes, however, the resulting parameterization code is quite compli- 
cated — many mixins need to be composed with others in a complex fashion. This 
introduces scalability problems (namely, extensions that instantiate template param- 
eters can be of length exponential to the number of mixins composed — see [24]). To 
make the approach more practical by reducing its complexity, mixin layers were 
introduced. Because mixin layers are an incremental improvement of the VanHilst 
and Nothin method, we only discuss implementing collaboration-based designs 
using mixin layers. 
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Mixin layers [24][25][26] are a particular form of mixins. They are designed with the 
purpose of encapsulating refinements for multiple classes. Mixin layers are nested 
mixins such that the parameter of an outer mixin determines the parameters of inner 
mixins. The general form of a mixin layer in C++ is: 

template <class NextLayer> 

class ThisLayer ; public NextLayer { 

public : 

class Mixinl ; public NextLayer Mixinl { ... }; 
class Mixin2 ; public NextLayer Mixin2 { ... }; 

}; 



Mixin layers are a result of the observation that a conceptual unit of functionality is 
usually neither one object nor parts of an object — a unit of functionality may span 
several different objects and specify refinements (extensions) to all of them. All such 
refinements can be encapsulated in a single mixin layer and the standard inheritance 
mechanism can be used for composing extensions. 

This property of mixin layers makes them particularly attractive for implementing 
collaboration-based designs. Each layer captures a single collaboration. Roles for all 
classes participating in a collaboration are represented by inner classes of the layer. 
Inheritance works at two different levels. First, a layer can inherit entire classes from 
its superclass (i.e., the parameter of the layer). Second, inner classes inherit members 
(variables, methods, or even other classes) from the corresponding irmer classes in 
the superclass layer. This dual application of inheritance simplifies the implementa- 
tion of collaboration-based designs, while preserving the benefits of the VanHilst and 
Notkin method. An important source of simplifications is that irmer classes of a 
mixin layer can refer unambiguously to other inner classes — the layer acts as a 
namespace. 

We illustrate our point with an example (presented in detail in [24]) of a collabora- 
tion-based design and its mixin layers implementation. (Full source code is available, 
upon request.) This example presents a graph traversal application and was examined 
initially by Holland [16] and subsequently by VanHilst and Notkin [29]. This appli- 
cation defines three different algorithms on an undirected graph, all implemented 
using a depth-first traversal: Vertex Numbering numbers all nodes in the graph in 
depth-first order. Cycle Checking examines whether the graph is cyclic, and 
Connected Regions classifies graph nodes into connected graph regions. The applica- 
tion has three distinct classes: Graph, Vertex, and Workspace. The Graph class 
describes a container of nodes with the usual graph properties. Each node is an 
instance of the Vertex class. Finally, the Workspace class includes the application part 
that is specific to each graph operation. For the VertexNumbering operation, for 
instance, a Workspace object holds the value of the last number assigned to a vertex 
as well as the methods to update this number. 

As shown in Fig 1, we can decompose this application into five independent collabo- 
rations — one encompassing the functionality of an undirected graph, another encod- 
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Object Classes 




Fig 1 : Collaboration decomposition of the example application: A depth-first traversal of an 
undirected graph is specialized to yield three different graph operations. Ovals represent col- 
laborations, rectangles represent classes, their intersections represent roles. 



ing depth-first traversals, and three containing the specifics of each graph algorithm 
(vertex numbering, cycle checking, and connected regions). Note that each collabo- 
ration captures a distinct aspect of the application and each object may participate in 
several aspects. That is to say, each object may play several roles. For instance, the 
role of a Graph object in the ^"Undirected Graph” collaboration supports storing and 
retrieving a set of vertices. The role of the same object in the ""Depth First Tra- 
versal” collaboration implements a part of the actual depth-first traversal algorithm. 

By implementing collaborations as mixin layers, the modular design of Fig 1 can be 
maintained at the implementation level. For instance, the ""Vertex Numbering” col- 
laboration can be implemented using a layer of the general form: 

template <class Next> 
class NUMBER : public Next { 
public : 

class Workspace : public Next :: Workspace { 

... // Workspace role members 

}; 

class Vertex : public Next:: Vertex { 

... // Vertex role members 



Note that no role (nested class) is prescribed for Graph. A Graph class is inherited 
from the superclass of Number (the class denoted by parameter Next). 
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As shown in [24], such components are flexible and can be reused and interchanged. 
For instance, the following composition builds Graph, Vertex, and Workspace 
classes nested inside class CycleC that implement vertex numbering of undirected 
graphs using a depth-first traversal:^^ 

typedef DFT < NUMBER < DEFAULTW < UGRAPH > > > CycleC; 

By replacing NUMBER with other mixin layers we get the other two graph algorithms 
discussed. Many more combinations are possible. We can use the templates to create 
classes that implement more than one algorithm. For instance, we can have an appli- 
cation supporting both vertex numbering and cycle checking on the same graph by 
refining two depth-first traversals in order: 

typedef DFT < NUMBER < DEFAULTW < UGRAPH > > > NumberC; 
typedef DFT < CYCLE < NumberC > > CycleC; 

Furthermore, all the characteristics of an undirected graph are captured by the 
UGRAPH mixin layer. Hence, it is straightforward to apply the same algorithms to a 
directed graph (mixin layer DGRAPH interchanged for UGRAPH) 

typedef DFT < NUMBER < DEFAULTW < DGRAPH > > > NumberC; 

This technique (of composing source components in a large number of combinations) 
underlies the scalable libraries [3] design approach for source code plug-and-play 
components. 

3 Programming with C++ Mixins: Pragmatic Considerations 

Since little has been written about the pragmatics of doing component programming 
using C++ mixins (mixin classes or mixin layers), we feel it is necessary to discuss 
some pertinent issues. Most of the points raised below concern fine interactions 
between the mixin approach and C++ idiosyncrasies. Others are implementation sug- 
gestions. They are all useful knowledge before one embarks into a development 
effort using C++ mixins and could serve to guide design choices for future parame- 
terization mechanisms in programming languages. The C++ aspects we discuss are 
well-documented and other C++ programmers have probably also made some of our 
observations. Nevertheless, we believe that most of them are non-obvious and many 
only arise in the context of component programming — that is, when a mixin is 
designed and used in complete isolation from other components of the system. 

Lack of Template Type-Checking. Templates do not correspond to types in the C++ 
language. Thus, they are not type-checked until instantiation time (that is, composi- 



The DEFAULTW mixin layer is an implementation detail, borrowed from the VanHilst and 
Nothin implementation [29]. It contains an empty Workspace class and its purpose is to 
avoid dynamic binding by changing the order of composition. 

This is under the assumption that the algorithms are still valid for directed graphs as is the 
case with the original code for this example [16]. 
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tion time for mixins). Furthermore, methods of templatized classes are themselves 
considered function templates. Function templates in C++ are instantiated auto- 
matically and only when needed. Thus, even after mixins are composed, not all their 
methods will be type-checked (code will only be produced for methods actually ref- 
erenced in the object code). This means that certain errors (including type mis- 
matches and references to undeclared methods) can only be detected with the right 
template instantiations and method calls. Consider the following example: 

template <class Super> 

class ErrorMixin : public Super { 

public : 

void sort(FOO foo) { 

Super :: srot (foo) ; // misspelled 



If client code never calls method sort, the compiler will not catch the misspelled 
identifier above. This is true even if the ErrorMixin template is used to create 
classes, and methods other than sort are invoked on objects of those classes. 

Delaying the instantiation of methods in template classes can be used to advantage, 
as we will see later. Nevertheless, many common designs are such that all member 
methods of a template class should be valid for all instantiations. It is not straight- 
forward to enforce the latter part (“for all instantiations”) but for most practical pur- 
poses checking all methods for a single instantiation is enough. This can be done by 
explicit instantiation of the template class, which forces the instantiation of all its 
members. The idiom for explicit instantiation applied to our above example is: 

template class ErrorMixin<SomeFoo> ; 

When “Subtype of’ Does not Mean “Substitutable for”. There are two instances 
where inheritance may not behave the way one might expect in C++. First, con- 
structor methods are not inherited. Ellis and Stroustrup ([13], p.264) present valid 
reasons for this design choice: the constructor of a superclass does not suffice for 
initializing data members added by a subclass. Often, however, a mixin class may 
be used only to enrich or adapt the method interface of its superclasses without add- 
ing data members. In this case it would be quite reasonable to inherit a constructor, 
which, unfortunately, is not possible. The practical consequence of this policy is that 
the only constructors that are visible in the result of a mixin composition are the 
ones present in the outer-most mixin (bottom-most class in the resulting inheritance 
hierarchy). To make matters worse, constructor initialization lists (e.g., 
constrO : initl(l,2), init2(3) {} ) 

can only be used to initialize direct parent classes. In other words, all classes need to 
know the interface for the constructor of their direct superclass (if they are to use 



This wording, although used by the father of C++ — see [28], p.330 — is not absolutely 
accurate since there is no automatic type inference. 
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constructor initialization lists). Recall, however, that a desirable property for mixins 
is that they be able to act as components: a mixin should be implementable in isola- 
tion from other parts of the system in which it is used. Thus a single mixin class 
should be usable with several distinct superclasses and should have as few dependen- 
cies as possible. In practice, several mixin components are just thin wrappers adapt- 
ing their superclass’s interface. 

A possible workaround for this problem is to use a standardized construction inter- 
face. A way to do this is by creating a construction class encoding the union of all 
possible arguments to constructors in a hierarchy. Then a mixin “knows” little about 
its direct superclass, but has dependencies on the union of the construction interfaces 
for all its possible parent classes. (Of course, another workaround is to circumvent 
constructors altogether by having separate initialization methods. This, however, 
requires a disciplined coding style to ensure that methods are always called after 
object construction.) As a side-note, destructors for base classes are called automati- 
cally so they should not be replicated. 

Synonyms for Compositions. In the past sections we have used typedefs to intro- 
duce synonyms for complicated mixin compositions — e.g., 
typedef A < B < C > > Synonym; 

Another reasonable approach would be to introduce an empty subclass: 

class Synonym ; public A<B<C>>{ }; 

The first form has the advantage of preserving constructors of component A in the 
synonym. The second idiom is cleanly integrated into the language (e.g., can be tem- 
platized, compilers create short link names for the synonym, etc.). Additionally, it 
can solve a common problem with C++ template-based programming: generated 
names (template instantiations) can be extremely long, causing compiler messages to 
be incomprehensible. 

Designating Virtnal Methods. Sometimes C++ policies have pleasant side-effects 
when used in conjunction with mixins. An interesting case is that of a mixin used to 
create classes where a certain method can be virtual or not, depending on the con- 
crete class used to instantiate the mixin. This is due to the C++ policy of letting a 
superclass declare whether a method is virtual, while the subclass does not need to 
specify this explicitly. Consider a regular mixin and two concrete classes instantiat- 
ing it (a C++ struct is a class whose members are public by default): 

template <class Super> 
struct MixinA ; public Super { 

void virtual_or_not (FOO foo) { ... } 

In- 
struct Basel { 

virtual void virtual_or_not (FOO foo) { ... } 

... // methods using "virtual_or_not" 

}; 
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struct Base2 { 

void virtual_or_not (FOO foo) { ... } 

}; 



The composition MixinA<Basel> designates a class in which the method 
virtual_or_not is virtual. Conversely, the same method is not virtual in the com- 
position MixinA<Base2>. Hence, calls to virtual_or_not in Basel will call the 
method supplied by the mixin in the former case but not in the latter. 

In the general case, this phenomenon allows for interesting mixin configurations. 
Classes at an intermediate layer may specify methods and let the inner-most layer 
decide whether they are virtual or not. 

As we recently found out, this technique was described first in [12]. 

Single Mixin for Multiple Uses. The lack of template type-checking in C++ can 
actually be beneficial in some cases. Consider two classes Basel and Base2 with 
very similar interfaces (except for a few methods): 

struct Basel { 

void regular ( ) { ... } 

struct Base2 { 

void weird ( ) { ... } 

... // otherwise same interface as Basel 

}; 



Because of the similarities between Basel and Base2, it makes sense to use a sin- 
gle mixin to adapt both. Such a mixin may need to have methods calling either of 
the methods specific to one of the two base classes. This is perfectly feasible. A 
mixin can be specified so that it calls either regular or weird: 

template <class Super> 
class Mixin : public Super { 

public : 

void methl ( ) { Super :: regular () ; } 

void meth2 ( ) { Super :: weird () ; } 

}; 



This is a correct definition and it will do the right thing for both composition 
Mixin<Basel> and Mixin<Base2>! What is remarkable is that part of Mixin 
seems invalid (calls an undefined method), no matter which composition we decide 
to perform. But, since methods of class templates are treated as function templates, 
no error will be signalled unless the program actually uses the wrong method (which 
may be methl or meth2 depending on the composition). That is, an error will be 
signalled only if the program is indeed wrong. We have used this technique to pro- 
vide uniform, componentized extensions to data structures supporting slightly dif- 
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ferent interfaces (in particular, the red-black tree and hash table of the SGI 
implementation of the Standard Template Library [22]). 

Propagating Type Information. An interesting practical technique (also applicable 
to languages other than C-l-l-) can be used to propagate type information from a sub- 
class to a superclass, when both are created from instantiating mixins. This is a com- 
mon problem in object-oriented programming. It was, for instance, identified in the 
design of the P-l-t- language [23] (an extension of C-l-l- with constructs for compo- 
nent-based programming) and solved with the addition of the forward keyword. 
The same problem is addressed in other programming languages (e.g.. Beta [19]) 
with the concept of virtual types. 

Consider a mixin layer encapsulating the functionality of an allocator. This compo- 
nent needs to have type information propagated to it from its subclasses (literally, the 
subclasses of the class it will create when instantiated) so that it knows what kind of 
data to allocate. (We also discussed this example in detail in [25] but we believe that 
the solution presented here is the most practical way to address the problem.) The 
reason this propagation is necessary is that subclasses may need to add data members 
to a class used by the allocator. One can solve the problem by adding an extra param- 
eter to the mixin that will be instantiated with the final product of the composition 
itself In essence, we are reducing a conceptual cycle in the parameterization to a sin- 
gle self-reference (which is well-supported in C-l-l-). This is shown in the following 
code fragment: 

template <class EleType, class FINAL> 
class ALLOC { 
public : 

class Node { 

EleType element; // stored data type 
public : 

... // methods using stored data 

}; 



class Container { 
protected : 

FINAL: ;Node* node_alloc() { 
return new FINAL: :Node() ; 

} 

... // Other allocation methods 

}; 



template <class Super> 

class BINTREE : public Super { 

public : 

class Node : public Super: :Node { 

Node* parent_link, left_link, right_link ; 
public : 

... // Node interface 

}; 
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class Container : public Super :: Container { 
Node* header; // Container data members 
public : 

... // Interface methods 




class Comp : public BINTREE < ALLOC <int, Comp> > {/* empty */ 

}; 



Note what is happening in this code fragment (which is abbreviated but preserves 
the structure of actual code that we have used). A binary tree data structure is cre- 
ated by composing a BINTREE mixin layer with an ALLOC mixin layer. The data 
structure stores integer (int) elements. Nevertheless, the actual type of the element 
stored is not int but a type describing the node of a binary tree (i.e., an integer 
together with three pointers for the parent, and the two children of the node). This is 
the type of element that the allocator should reserve memory for. 

The problem is solved by passing the final product of the composition as a parame- 
ter to the allocator mixin. This is done through the self-referential (or recursive) 
declaration of class Comp. (Theoretically-inclined readers will recognize this as a 
fixpoint construction.) Note that Comp is just a synonym for the composition and it 
has to use the synonym pattern introducing a class (i.e., the typedef synonym 
idiom discussed earlier would not work as it does not support recursion). 

It should be noted that the above recursive construction has been often used in the 
literature. In the C++ world, the technique was introduced by Barton and Nackman 
[2] and popularized by Coplien [9]. Nevertheless, the technique is not mixin-spe- 
cific or even C++-specific. For instance, it was used by Wadler, Odersky and the 
first author [32] in Generic Java [7] (an extension of Java with parametric polymor- 
phism). The origins of the technique reach back at least to the development of F- 
bounded polymorphism [8]. 

Hygienic Templates in the C++ Standard. The C++ standard ([1], section 14.6) 
imposes several rules for name resolution of identifiers that occur inside templates. 
The extent to which current compilers implement these rules varies, but full con- 
formance is the best approach to future compatibility for user code. 

Although the exact rules are complicated, one can summarize them (at loss of some 
detail) as “templates cannot contain code that refers to ‘nonlocal’ variables or meth- 
ods”. Intuitively, “nonlocal” denotes variables or methods that do not depend on a 
template parameter and are not in scope at the global point closest to the template 
definition. This rule prevents template instantiations from capturing arbitrary names 
from their instantiation context, which could lead to behavior not predicted by the 
template author. 
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A specific rule applies to mixin-based programming. To quote the C++ standard 
(14.6.2), “if a base class is a dependent type, a member of that class cannot hide a 
name declared within a template, or a name from the templates enclosing scopes”. 
Consider the example of a mixin calling a method defined in its parameter (i.e., the 
superclass of the class it will create when instantiated): 

struct Base { 

void foo 0 { ... } 

}; 

void food { } 

template <class Super> 
struct Mixin : public Super { 

void which_one() { food; } // ; : foo 

}; 



Mixin < Base > test; 

That is, the call to foo from method which_one will refer to the global foo, not the 
foo method of the Base superclass. 

The main implication of these name resolution rules is on the way template-based 
programs should be developed. In particular, imagine changing a correct class defini- 
tion into a mixin definition (by turning the superclass into a template parameter). 
Even if the mixin is instantiated with its superclass in the original code, the new pro- 
gram is not guaranteed to work identically to the original, because symbols may now 
be resolved differently. This may surprise programmers who work by creating con- 
crete classes and turning them into templates when the need for abstraction arises. To 
avoid the potential for insidious bugs, it is a good practice to explicitly qualify refer- 
ences to superclass methods (e.g.. Super : ; foo instead of just foo). 

Compiler Support. Most C++ compilers now have good support for parameterized 
inheritance (the technique we used for mixins) and nested classes. We have encoun- 
tered few problems and mostly with older compilers when programming with C++ 
mixins. In fact, most of the compiler dependencies are not particular to mixin-based 
programming but concern all template-based C++ programs. These include limita- 
tions on the debugging support, error checking, etc. We will not discuss such issues 
as they are time-dependent and have been presented before (e.g., [10]). Note, how- 
ever, that mixin-based programming is not more complex than regular template 
instantiation and typically does not exercise any of the “advanced” features of C++ 
templates (type inference, higher-order templates, etc.). Overall, the compiler sup- 
port issues involved in mixin-based programming are about the same as those arising 
in implementing the C++ Standard Template Library [27]. 
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4 Related Work 

Various pieces of related work have been presented in the previous sections. We 
cannot exhaustively reference all C++ template-based programming techniques, but 
we will discuss two approaches that are distinct from ours but seem to follow paral- 
lel courses. 

The most prominent example is the various implementations of the STL. Such 
implementations often exercise the limits of template support and reveal interac- 
tions of C++ policies with template-based programming. Nevertheless, parameter- 
ized inheritance is not a part of STL implementations. Hence, the observations of 
this paper are mostly distinct from the conclusions drawn from STL implementation 
efforts. 

Czamecki and Eisenecker’s generative programming techniques [10][11] were used 
in the Generative Matrix Computation Library (GMCL). Their approach is a repre- 
sentative of techniques using C++ templates as a programming language (that is, to 
perform arbitrary computation at template instantiation time). What sets their 
method apart from other template meta-programming techniques is that it has simi- 
lar goals to mixin-based programming. In particular, Czamecki and Eisenecker try 
to develop components which can be composed in multiple ways to yield a variety 
of implementations. Several of the remarks in this paper are applicable to their 
method, even though their use of mixins is different (for instance, they do not use 
mixin layers). 

5 Conclusions 

We presented some pragmatic issues pertaining to mixin-based programming in 
C++. We believe that mixin-based techniques are valuable and will become much 
more widespread in the future. Mixin-based programming promises to provide reus- 
able software components that result into flexible and efficient implementations. 

Previous papers have argued for the value of mixin-based software components and 
their advantages compared to application frameworks. In this paper we tried to 
make explicit the engineering considerations specific to mixin-based programming 
in C++. Our purpose is to inform programmers of the issues involved in order to 
help move mixin-based programming into the mainstream. 
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Abstract. Software evolution demands continuous adaptation of soft- 
ware systems to continuously changing requirements. Our goal is to cope 
with software evolution by automating program transformation and sys- 
tem reconfiguration. We show that this can be achieved with a static 
metaprogramming facility and a library of suitable metaprograms. We 
show that former approaches of program transformations are not suffi- 
cient for large object oriented systems and outline two base transforma- 
tions that fill the gap. 



1 Introduction 

Many software systems have grown enormously large in the last decades. Require- 
ments for these systems have changed continuously, and still do. Consequently, 
the software needs to be adapted all the time: software evolves. However, chang- 
ing software is a complex and error prone task and is likely to produce high 
costs. One reason for this are lots of dependencies in typical programs, which 
make them hard to understand. Moreover, the effects of changes to a system 
are hard to predict. This is especially true for large changes like system restruc- 
turing, addition, removal and exchange of components and subsystems as well 
as composition of components. But even small modifications like changing the 
signature of a single method may show effects throughout the whole system: 
for instance, a changed method parameter type can not only influence all ref- 
erences to that method but also all references to overloaded methods which in 
turn might be defined in any subclass. To deal with those effects, we require 
global information about the program and must hence operate in the large. We 
provide further examples in sections 2 and 3. 

Software evolution means transforming programs. Our group developed a 
static metaprogramming facility called COMPOST that provides a library of 
useful analyses and transformations. We found metaprogramming to be a natural 
and very promising strategy to carry out changes consistently and thus support 
both software evolution and software construction. 

To cope with software evolution and construction, software must be easy to 
change. We therefore focus on software changes and start with a concrete sce- 
nario, the evolution from JDK 1.0 to JDK 1.3 (section 2). After that, we identify 
some program transformations that are needed for our purposes, especially to 
deal with imperative object oriented programs (section 3). We then outline the 
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infrastructure needed to actually perform those changes (section 4). Finally, we 
discuss related work and possible uses of metaprogramming in these areas, which 
we cannot cover in detail here (section 5). We conclude with our experiences and 
suggestions for future work (section 6). 



2 Software Evolution Scenario 

Assume that a new version of a component offers improved behaviour, but also 
comes with a changed interface. In order to use the revised version, all using 
contexts of this component have to be updated. Now assume that the programs 
that have been using the old component are also maintained. Then, binary com- 
patibility is not sufficient. Instead, the programs must be brought to the state 
of the art at source level. 

Example 1 (JDK Update). There have been about 300 changes in the API 
of the standard Java libraries up to now — a complete list is available at 
http: //java. sun. com/j2se/l . 3/docs/api/deprecated-list .html. These interface 
changes affect all pieces of Java software that rely on the now “deprecated” features; 
we classify these changes as follows and will study them in detail later on: 

® Local changes of signatures (name or parameters), e. g. in java. awt. Component: 

— move(int, int) ^ setLocation(int, int) 

— disable() — > setEnabled(false) 

— add(String, Component) — » add(Component, Object) 

® Change of the location of features, e.g. in java. awt: 

— Frame.CROSSHAIR.CURSOR Cursor.CROSSHAIR.CURSOR 

— Frame. getCursorType() — > Component. getCursor().getType() 

@ Removal or replacement of features, e.g. 

— java. awt. Component. getPeer() — » 0 (deleted) 

— java.io.StringBufferInputStream java.io.StringReader 

® Change of an entire framework, using multiple design patterns, such as the new 
event handling framework that replaces a template method based solution by an 
observer pattern, which concerns some dozen classes in several packages. 

How can we automate an update for a large system? The solution is at hands: 
Let the programmer of the component write a “smart patch” metaprogram that 
is able to update arbitrary programs. 

The scenario illustrates what we consider metaprogramming in the large. In 
this setting, we have a big advantage, as some of the semantics of the system 
to change is known. Assuming that the old versions of the components have 
been used correctly allows to assert many predicates that would have been very 
hard to find out by analyses, such as object lifetime dependencies. After some 
basic transformations have been detailed, we illustrate how this knowledge can 
be used in the second part of the example. 
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3 New Program Transformations 

Program transformations have been investigated for years (for a good overview 
see [CEOO]). We discuss how goals and languages have changed since then and 
now require further transformations. We then sketch an abstract program model 
to deal with object oriented languages, and finally discuss two fundamental trans- 
formations based upon that model. 

3.1 Classical Program Transformations Improved 

The primary research focus of classical program transformations was program 
refinement and optimisation, as well as the creation of extensible languages. The 
most important achievements are a set of base transformations such as add def- 
inition, fold, unfold, function generalisation and specialisation, and techniques 
to resolve recursions. These transformations were based upon a functional pro- 
gramming model and local contexts only, to avoid the problems introduced by 
states and scopes. However, states and scopes are inherent parts of imperative 
object oriented languages. While the known base transformations reappear in 
more specific forms to suit the rules of the concrete language, further base trans- 
formations are required to cover states and scopes. 

Transformations come with a set of constraints restricting their use, e.g. 
prohibition of ambiguous names. Single transformations often lack constructive 
strategies how to fulfil these conditions; usually, only a combination of transfor- 
mations is applicable. Thus, the goal is to collect sufficient information about 
the context to derive applicable chains of transformations to achieve the desired 
effects. 

3.2 Nested Feature Model for Object Oriented Languages 

The notions of component interfaces 
and scopes are somewhat dependent 
on the concrete language. There- 
fore, we introduce an abstract pro- 
gram model consisting of hierarchi- 
cally nested features such as classes, 
variables, and methods, which may 
define a scope (see Figure 1). We 
may regard packages, constructors, at- 
tributes or parameters as special cases 
of classes, methods, or variables. To 

complete this simplistic model, we distinguish declarations of features and ref- 
erences to features which are contained within a scope, and assume that each 
feature has at most one definition, but arbitrarily many references. The exact 
rules for feature declarations, feature nesting, and feature reference resolution 
(including feature visibilities, overloading, and inheritance) are part of the lan- 
guage specification. In our experiments, we used Java as a representative object 
oriented language. 
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3.3 A Transformation to Deal with States 

The feature model has no visible states to manipulate directly. However, the 
state transitions defined by variable and method accesses can be used to bring 
in new states. These accesses may be embedded into arbitrary expression terms. 
To avoid side effects in the partial expressions that are evaluated prior to the 
feature access, we need a transformations that shifts these parts and flattens the 
syntax tree of the expression term. The following example illustrates this. 

Example 2 (Expression Elattening) . Consider the expression x = f(g(y), h(k(z)) and 
assume we want to insert some code before the very moment when f is accessed. The 
transformation should produce: int i = g(y); boolean b = h(k(z)); /* some new code */ 
X = f(i, b); — now the reference to f is the first expression that is evaluated in the state- 
ment. Note that the types of g and h cannot be derived from the original expression, 
and that it was not necessary to flatten k. 

The transformation takes a reference to a variable or method as argument 
and flattens the top-level expression term this reference is contained within: 

1. Collect all expressions that become evaluated before the relevant feature 
access and that may have side effects. 

2. For each largest expression e in the order of evaluation: 

(a) Calculate the type r of e 

(b) Create a new unambiguous temporary variable v of type r and insert 
the declaration of v in front of the the top-level expression. 

(c) Insert e as initialisation code of the variable. 

(d) Substitute the original occurrence of e with a new reference to v. 

This code transformation allows to modify arbitrary accesses to features by 
simple insertion of new code. It is easy to see how it can be used for instrumen- 
talisation or to exchange method calls by other means of communication. 

Note that this transformation requires context sensitive information to derive 
the types of expressions, and to choose an unambiguous variable name. 

3.4 A Transformation to Deal with Scopes 

The most general and basic transformation that is able to reconfigure inter- 
faces is move feature declarations} The transformation consists of two subtasks, 
which are also base transformations: creating new feature declarations in the 
new scope, and deletions of old declarations. Note that we explicitly allow in- 
heritance, feature hiding, overloading, and combinations thereof. 

1. Insert feature declarations in the new scope 

(a) Check if a declaration is forbidden due to ambiguous names in the new 
scope, or incompatible modifiers or signatures while attempting to over- 
write a feature of identical name in a superclass. If so, rename the feature, 
or optionally change modifiers in the superclass. 

^ Special cases of this transformation are already depicted in [FBB''’99] as refactorings. 
The version presented here is more general and more detailed. 
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(b) Check if a definition hides less general definitions that are overloaded or 
overwritten. If so, rename the new features, or strengthen qualification 
of references to the hidden feature (add type casts or access qualifiers). 

2. Remove feature declarations from the old scope 

(a) Check if there are references from the moved features to other features in 
the old scope. If so, redirect them to the old scope using a proper access 
path. If no predefined access path is available, create a new one by adding 
an additional parameter to each feature. If necessary, adapt visibilities 
of the used features to allow access from the new scope. Alternatively, 
compute a self-dependent feature group and ask the user if this group 
shall be moved instead.^ 

(b) Check if there are external references to the features in the old scope. If 
so, redirect them to the new scope, by using a proper access path. 

This transformation allows to reconfigure system interfaces arbitrarily. If one 
reads design patterns as instructions how to change a system rather than creating 
one from scratch, then many of them are built upon this transformation. 

Finding access paths is a key issue in this transformation when the scopes 
are classes. In many cases, access paths are predefined, either by the caller of the 
transformation, or if features are global, or the scopes are nested or inherited (see 
Figure 2) — then, the underlying objects are either accessible from anywhere, 
identical, or share the same life time. 
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Fig. 2. Feature Movement Schemes - General Case and Two Specialisations 
(below) — Note that the admissible feature types of x, y and z may vary. 

In the general case, no predefined access path is available. Sometimes, it is 
possible to identify a unique path candidate that is free of possible side effects. If 
there are several solutions, the choice is a genuine design decision and should be 
left to the user. If there is not even a possible candidate, the transformation can 
attach a new attribute and insert some initialisation code for the target object, 
if the initialisation is not ambiguous. 

We learn two lessons from this example: First, user interaction is required 
when analyses fail and design alternatives must be chosen. Second, a non-trivial 
transformation requires a lot of subtasks which have to be addressed. Figure 3 
illustrates some auxiliary functions. 

This is the reason why moving a set of declarations can be easier than a single one. 



2 





Metaprogramming in the Large 183 



Insert Feature Declaration 




Fig. 3. Transformation Subtasks. 

Example 3 (JDK Update Revisited). Equipped with the transformations just pre- 
sented and additional knowledge about concrete components that we have to deal with, 
we can now try to find a metaprogram that would have saved enormous amounts of 
programming time. . . 

(b Local changes of signatures: Renaming is a task similar to addition of a new feature, 
while a change of parameters requires a change of arguments on the caller side — 
flattening the expressions that might have side effects is the most difficult part. 

® Change of the location of features: The examples given are easy to handle with 
feature moves as we have to deal with global constants, or already know a proper 
replacement access path. 

@ Removal or replacement of features: In the case of the deprecated Peer classes, there 
is no obvious update. However, the only proper use of the peers has been to check 
for consistent initialisation state. We can attempt to pattern match for c.getPeer() ! = 
null and replace this expression by c.isDisplayable(). The second example concerning 
the replacement of an improper Unicode stream, is also hard to resolve, as different 
data types (byte vs. char) are involved. But again, all meaningful uses should not rely 
on concrete bit counts, so a simple reassignment of methods to their counterparts is 
sufhcient. Only one method features a different signature: read(char[],int,int) replaces 
read(byte[],int,int). Typical uses of the old version will have translated the byte array 
into a string or a character array anyway, but to ensure compatibility, we would have 
to insert some translation code. We have to attach some glue code to calls of the form 
in.read(arr, off, len) in order to obtain: 

char[] tmp = new char[arr. length]; 
in.read(tmp, off len); 

System. arraycopy(new String(tmp, off len).getBytes(), 0, arr, off len); 

There are some details to get right, such as adding a new statement block when nec- 
essary and getting rid of side effect expressions in access paths. 

® Change of an entire framework: Beginning with JDK 1.1, event handling changed 
to an observer pattern replacing redefined methods of the corresponding components. 
The following code snippets show a simple solution obtained by some feature moves. 
The former action method in class C 

public boolean action(Event e, Object what) { 

/ / action code here 

return super. action(e, what); // or other expression 

} 

is transformed into a new attribute declaration in C 
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Action Listener actions = new Action Listener() { 
public void actionPerformed(ActionEvent e) { 

// user code from super. action goes here 

// action code here; ‘what’ redirected to e.getActionCommand() 

//no return value here - no need to consume events 

} 

}; 

plus an additional call addActionListener(actions); in each constructor of C. 

Example 3 shows that an automatic update is actually possible while human 
interaction is necessary only in few cases. 

In the following section, we discuss the specific requirements and challenges 
for a technical infrastructure supporting the construction of metaprograms. 

4 Object Oriented Metaprogramming Realized 

The general procedure of source 
to source transformations is il- 
lustrated in Figure 4: A com- 
piler frontend produces a meta- 
model from the sources, which 
includes all syntactic data nec- 
essary for the pretty printer to 
reproduce the sources later on. 

Transformations can use any 
derived information to change 
the syntactic parts. 

4.1 Requirements for Metaprogramming Systems 

Metaprograms must cover some important requirements, and these must be con- 
sidered when supplying tools to facilitate the construction. 

Readability: A metaprogramming system changes source codes. The result 
should obey coding conventions and should not destroy formatting. In this set- 
ting, blank spaces become important and must be protected. In general, “code 
bloat” should be kept to a minimum. 

Scalability: In contrast to compilers, metaprogramming systems change the syn- 
tactic base of the metamodel during runtime. Hence, when used in an interac- 
tive and incremental environment, information derived from the syntactic model 
must also be updated incrementally. To retain scalability, it is not feasible to 
invalidate the whole model after a change. Instead, it is necessary to carefully 
analyse change impacts to prevent transitive propagation throughout the model. 
For instance, a change of a feature declaration may change the assignment of all 
references to the feature and all hidden ones. 

Note that the smart patch scenario is not necessarily incremental — usually it is 
possible to do all necessary analyses in advance and delay the transformations. 




Fig. 4. Source Level Transformations. 
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Extensibility: There are lots of possible transformations motivated by high level 
design problems. A library of metaprograms is therefore unlikely to be complete, 
but it should be extensible. A proper system design should facilitate the creation 
of metaprograms. For instance, the metamodel interface should hide the process 
of information retrieval, offer navigational links, and provide meaningful abstrac- 
tions over the concrete representations. Auxiliary transformations, analyses and 
code snippet generators should become part of the library. 

4.2 Limitations of Metaprogramming Systems 

Source level metaprogramming suffers from some general problems. 

Source code hygiene and especially comment assignment are an issue. Com- 
ments will not always be assigned properly and may move with the wrong ele- 
ment. Also, it will not always be possible to avoid some redundant code. 

In general, comments and documentation do not obey a formal semantics 
and hence cannot be maintained automatically. However, it is possible to auto- 
mate changes of designated, formal parts. For instance, the Java documentation 
conventions require to tag feature references with Osee or ©link making them 
accessible for analysis and transformations. 

Analyses are conservative and may fail to derive enough information to do 
certain decisions automatically. Then, interaction with the human programmer 
is needed. Sometimes an alternative approach can be found, which might be far 
from perfect, but functional — e. g. to insert a new attribute and ignore an un- 
certain candidate. Possible improvements could then be offered in an interactive 
session after the main transformations have taken place. 

4.3 COMPOST - An Implementation 

We have been constructing a metaprogramming system for the Java program- 
ming language called COMPOST (COMPOsition SysTem). The system is avail- 
able from http://i44www.info.uni-karlsruhe.de/~compost and consists of 
nearly 600 classes. It contains a Java frontend providing facilities to resolve 
feature references and for cross reference information. The pretty printer is con- 
figurable to fit specific conventions and is able to reproduce (or improve upon) 
the original input and to insert new code seamlessly. We have built a library 
of auxiliary functions and transformations also containing the auxiliaries from 
Figure 3. 

Transformations are responsible for the maintenance of the abstract syntax 
model, while the pretty printer handles the concrete syntax, and lookup services 
update the derived semantic information. Transformations report the syntactic 
effects of their changes, and this information is interpreted by a responsible 
service as soon as new semantic information is required. 

4.4 Experiences 

Memory consumption of a metaprogramming system is high, but does not exceed 
that of an optimising compiler, and seems to scale with the size of the program. 
We were able to analyse 3000 classes on a state of the art workstation. 
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Naive incremental model updates are sufficient for a handful of classes only, 
but more sophisticated algorithms should be able to handle a big number of 
classes, as some sparse experiments in literature have indicated [BS86] [Wag98]. 

We have chosen Java as our primary target language, but metaprogramming 
should be applicable for all programming languages. In general, languages pro- 
viding many static predicates allow to do more decisions automatically based on 
the additional knowledge represented by these predicates. 

5 Related Work 

We already compared metaprogramming with program transformation systems 
in section 3. Now we discuss related work in the area of software evolution. 

Extreme programming (XP) [Bec99] is a development process to support 
evolutionary programming. XP relies on testing to assure the correctness of 
changes. Verified metaprograms would no longer require additional tests. 

The base mechanics of XP are refactorings [FBB+99]. Refactorings are pro- 
gram transformations that optimise for readability. To create sophisticated tool- 
ing, the strategies given in the catalogues are still too vague and heavily rely 
on the programmer. Compiler construction research can provide solutions for 
many problems, but we need a more formal background to do so. The feature 
movement transformation is an attempt to generalise many special cases. 

While refactorings are too local to have a meaningful motivation by them- 
selves, design patterns [GHJV95] are complete development processes leading 
from a single design problem down to implementations. In many cases, the pat- 
tern can be regarded as an instruction to change software, using a series of 
refactorings and similar transformations [Zim97]. If this is possible, then we can 
derive a metaprogram instantiating the pattern. For instance, moving features 
is fundamental to create new levels of indirection widely used in patterns. 

Architectural systems [SDZ95] allow to exchange connectors representing a 
communication association. Similarly, composition languages [NM95] focus on 
the wiring of software components. Metaprograms can easily exchange connec- 
tors or compose programs avoiding special languages. 

Generic programming allows to bind parameters embedded in the code with 
concrete syntactic content. Metaprogramming can simulate the instantiation of 
a parameter and even attempt to identify the replacement locations even if they 
were not tagged as parametric. 

Aspect oriented programming [KIL’''97] or multidimensional separation of 
concerns [OT99] intend to offer multiple views on the (abstract) program that 
might not be identical to the primary functional modularisation. Each aspect, 
or concern, can be changed separately, and must then be merged, or woven, 
with the other aspects to derive the final model. This weaving process can be 
implemented by a metaprogram. 

In this work we concentrated on the support of transformations for refactoring 
software systems rather than the configuration of system families. The latter is 
subject of the generative programming paradigm [GEOO] . 
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6 Conclusion 

We showed that program transformations realized by metaprograms significantly 
facilitate consistent and correct changes and that this approach is a well suited 
technique for software evolution. Furthermore, we demonstrated that metapro- 
gramming fits seamlessly into emerging design techniques such as design pat- 
terns, separation of concerns, and refactorings. 

We identified two important basic transformations that are necessary for 
reconfiguring large imperative object oriented systems. We also showed that 
metaprogramming is feasible for real world languages with the right tool sup- 
port. We described the infrastructure necessary for source to source metapro- 
gramming. COMPOST is an implementation of such an infrastructure. 

Future work comprises the identification of further base transformations for 
object oriented programs as well as fast incremental model updates since there 
is still little experience in this area. 
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Abstract. Template techniques in C++ allow a modest degree of gen- 
erative programming-, creating specialized code for specialized problems. 
This use of templates has been controversial; indeed, one of the oft-cited 
reasons for migrating to Java is that it provides a simpler language, free 
of complexities such as templates. The essence of generative program- 
ming in C++ is not templates - the language feature - but rather the 
underlying algorithms in the compiler (template instantiation) which un- 
intentionally resemble an optimization called partial evaluation [12,18]. 
By devising a partial evaluator for Java, we reproduce some of the gen- 
erative programming aspects of C++ templates, without extending the 
Java language. The prototype compiler, called Lunar, is capable of doing 
“expression templates” in Java to optimize numerical array objects. 



1 Introduction 

Good numeric performance for Java is mostly limited to code written in what 
might be called JavaTran (Java/FORTRAN) style: Fortran 77-like code sur- 
rounded by class declarations. A typical example is: 

public class DoArrayStuff { 

public static apply (float [] w, f loat [] x, 
f loat [] y, f loat [] z) 

{ 

for (int i=0; i < w. length; ++i) 
w[i] = x[i] + y[i] * z [i] ; 

} 

} 

This is not much of an improvement over Fortran 77. It would be nice if we could 
use the object-oriented features of Java in performance-critical code. Then we 
could write high-level code using objects representing arrays, matrices, tensors, 
and the like. Suppose we had a package which provided efficient numerical array 
objects; with operator overloading as proposed by the Java Grande committee 
[6], we could have Fortran-90 style array notation. Using such a package, the 
DoArrayStuff code above could be written as: 



G. Butler and S. Jarzabek (Eds.): GCSE 2000, LNCS 2177, pp. 188-200, 2001. 
© Springer- Verlag Berlin Heidelberg 2001 




Just When You Thought Your Little Language Was Safe 189 



public static applyCArray w, Array x. Array y, 

Array z) 

{ 

w = X + y * z; 

} 

Without operator overloading, one would write: 
w. assignCx.plus (y. times (z) ) ) 

This is an improvement over JavaTran (Java in Fortran 77 style). However, can 
this Array class be implemented efficiently? This is an old question, with four 
well-known implementation choices: 

1 . Compiler extension: Write a compiler or preprocessor which recognizes a 
particular array class, and optimizes it using special semantics. This is the 
intent of the valarray class in the C-|— I- standard, and of the ROSE prepro- 
cessor for C-I--I- [1]. In Java, this approach is used by the Ninja compiler [11], 
which recognizes a special set of array classes and performs optimizations 
for them using “semantic inlining” . 

2. Pairwise evaluation: Each subexpression such as y. times (z) returns an 
intermediate Array result. The extra loops, memory allocation, and memory 
movement make this inefficient. Automatically fusing these loops is more 
difficult than the equivalent Fortran or C loop fusion problem, because the 
array pointers and loop bounds are fields of array objects. So far, no com- 
mercial compiler has been able to eliminate the temporary arrays. 

3. Deferred evaluation: In this approach, expressions such as y. times (z) 
construct a parse tree of an expression as a data structure. When a parse 
tree is assigned to an array, a match can be sought in a library of commonly 
used array expressions. This requires considerable run-time overhead, and if 
the expression is not found, one must revert to temporary arrays. 

4. Expression templates [16]: This C-|— I- technique is similar to deferred 
evaluation, in that a parse tree of the expression is created. The parsing is 
done at compile time using C-I--I- templates, by encoding the parse tree as 
a template type.^ When a parse tree is assigned to an array, a function like 
this is called: 

template<class T> 

void assign (Array w, Expr<T> expr) 

{ 

for (int i=0; i < n; ++i) 
w.data[i] = expr .eval(i) ; 

} 

For each array position i, expr.eval(i) traverses the parse tree, evaluating 
the expression. C-|— I- semantics require that this traversal be done at compile 

For example, x+y*z might be parsed as the template type 
Expr <Arr ay , Plus , Expr <Arr ay , Times , Arr ay> > 
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time, so the resulting loop is efficient. This technique is the basis of C++ 
libraries such as Blitz++ [17] and POOMA [8]. 

Of these approaches, expression template-based array libraries have been popular 
for C++, since they deliver efficiency without compiler extensions. 

It turns out that templates in C++ are strikingly like partial evaluation. A 
partial evaluator takes a program, performs the operations which depend only 
on known values, and outputs a specialized program [7]. The standard example 
is shown in Figure 1. 



float pow (float x, int n) 

{ 

if (n == 0) 



return 1.0; 
else 

return x * pow(x,n-l); 

} 



float a, b; 
a = pow(b,3) ; 

(a) Some code 



// pow has been specialized for n=3 
float pow__3 (float x) 

{ 

return x * x * x; 

} 



float a, b; 
a = pow__3(b); 



(b) After partial evaluation 



Fig. 1. Partial Evaluation Example. 



CH — h requires that all template parameters be known at compile time. When 
a template parameter is given by an expression (for example, Vector<3+8>), 
that expression must be evaluated at compile time. C++ templates effectively 
require that a partial evaluator be built into the compiler to evaluate template 
expressions [12,18]. It is this partial evaluation which makes possible expression 
templates and other template-based optimizations. 

So here is a thought. Perhaps if we implemented a partial evaluator for Java, 
we could get the same performance benefits one gets from templates in C++, 
without templates. 

This turns out to be at least partly true. We demonstrate a partial evaluator 
for Java that can be used to implement “expression templates” and similar 
performance-enhancing techniques from C++. We do this without any language 
extensions; we do not use GJ or other genericity proposals for Java. 

1.1 Structure of this Paper 

We start by considering an analogue of the C++ “expression templates” tech- 
nique for Java (Section 2). This gives users array notation, at the cost of very 
poor performance. When this code is partially evaluated, we obtain performance 
similar to code written in JavaTran style (Section 3). In Section 4, we overview 
the Lunar compiler and summarize the optimizations performed. We point out 
some shortcomings of our compiler and some related work in Section 5. 
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2 A Java Version of “Expression Templates” 

Our goal is to write a class Array which provides a 1-D array of floats. Users of 
this class will write code such as: 

int n = 1000; 

Array w = new Array (n) ; 

Array x = new Array (n) ; 

Array y = new Array (n) ; 

Array z = new Array (n) ; 

w = x + y*z; //eui array expression 

Since we do not yet have overloaded operators in Java, we will write w=x+y*z 
like this: 

w. assignCx.plus (y. times (z) ) ) ; 

Array expressions will be evaluated for every element in the array; the equivalent 
JavaTran version of w=x+y*z is: 

float [] w,x,y,z; II ... 

for (int i=0; i < n; ++i) 
w[i] = x[i] + y[i] * z [i] ; 

We start by creating a highly inefficient mechanism for evaluating array expres- 
sions which resembles the “expression templates” technique of C-I--I-. Then we 
will see that a suitable optimizing compiler can automatically turn this into the 
equivalent JavaTran implementation. More generally, such a optimizer has the 
potential to duplicate many of the generative programming features of C-| — h 
templates. 



Expr BinaryOperator 

T T 

Array BinaryOpExpr Plus Times 

Fig. 2. Inheritance Diagram for the Array Classes. 



For our pseudo- “expression templates” implementation, we will need objects 
representing array expressions. We will use the class hierarchy in Figure 2: Expr 
is a common base class of array expressions, and Array is itself an expression. 
To represent an expression such as y*z (or y. times (z)), we use an instance of 
a class BinaryOpExpr (short for Binary Operator Expression). BinaryOpExpr 
contains two expressions and a pointer to a binary operator object; to create an 
object representing y*z, we could write 

Expr expr = new BinaryOpExpr (y, z, new TimesO); 
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where Times is an object representing multiplication. The definitions of these 
classes are shown in Figures 3 and 4. 

The expression base class Expr defines an abstract method float evalCint 
i) which evaluates an array expression at a single array index i. Hence we can 
assign an expression to array using this method of class Array: 

public class Array extends Expr { 

public void assign(Expr e) 

{ 

for (int i=0; i < length; ++i) 
data[i] = e.eval(i); 

} 

} 



public class Array extends Expr { 
float data [] ; 
int length; 

public Array(int n) 

{ 

data = new float [n] ; 
length = n; 

} 

public float evaKint i) 

{ 

return data[i]; 

} 

public void set (int i, float value) 

{ 

data[i] = value; 

} 

public void assign (Expr e) 

{ 

int t = length; 
for (int i=0; i < t; i=i+l) 
data[i] = e.eval(i); 

} 

} 



public abstract class Expr { 

public abstract float evaKint i); 

public Expr plus (Expr b) 

{ 

BinaryOperator plus = new PlusO; 
return new BinaryOpExpr (this ,b ,plus) ; 

} 

public Expr times (Expr b) 

{ 

BinaryOperator times = new TimesO; 
return new BinaryOpExpr (this ,b , times) ; 

} 

} 

public class BinaryOpExpr extends Expr { 
Expr a, b; 

BinaryOperator op; 

public BinaryOpExpr (Expr _a, Expr _b, 
BinaryOperator _op) 

{ 




op - _op ; 



} 

public float evaKint i) 

{ 

return op . apply(a. eval (i) ,b . eval(i) ) ; 

} 

} 



Fig. 3. The Array Class, and Parse Tree Classes Expr and BinaryOpExpr. 



Figure 5 shows a test program which exercises the Array class. It initializes some 
arrays, then evaluates the expression w=x+y*z. The method which assigns the 
expression to w is Array, assign (Expr): 
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public abstract class BinaryOperator { 

public abstract float applyCfloat a, float b) ; 

} 

public class Plus extends BinaryOperator { 
public float applyCfloat a, float b) 

{ 

return a+b; 

} 

} 

public class Times extends BinaryOperator { 
public float applyCfloat a, float b) 

{ 

return a*b; 

} 

} 



Fig. 4. The BinaryOperator Base Class and Two Subclasses, 

public class Test { 

public static void mainCjava.lang. String!] args) 

{ 

// Create some arrays 
int n = 12345; 

Array w = new Array Cn) ; 

Array x = new Array Cn) ; 

Array y = new Array Cn) ; 

Array z = new Array Cn) ; 

// Initialize with data 
for Cint i=0; i < n; i=i+l) 

{ 

X. set Ci , i*0 . 33f ) ; 

y . set Ci , 10 . Of+i) ; 

z . set Ci , 100 . Of *i) ; 

} 

// With operator overloading , this would be 
II w=x+y*z 
w. assignCx.plus Cy. times Cz) ) ) ; 

} 

} 



Fig. 5. Test Code for the Array Class. 
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public void assign (Expr e) 

{ 

for (int i=0; i < length; i=i+l) 
data[i] = e.eval(i); 

} 

This method loops through the array, evaluating the array expression for each 
i, and storing the result in w.data[i] . To evaluate each e.eval(i), six virtual 
function calls, three bound checks, and numerous pointer indirections are re- 
quired. Not surprisingly, performance is quite poor with typical Java compilers. 
This loop has been benchmarked at 0.7 Mflops using the Kaffe JIT compiler on 
a 300 MHz sparcvO; that is roughly 1 flop every 428 clock cycles. 

With partial evaluation, all of this inefficiency can be removed. We have 
implemented a prototype compiler called Lunar that compiles Java to an inter- 
mediate form, partially evaluates it, and emits C code. This is then compiled 
to machine code using a C compiler. When applied to the example program 
(Figure 5), Lunar performs these optimizations: 

— Constant and copy propagation through the heap, resolving (when possible) 
virtual method calls and pointer indirection at compile time. 

— Inlining virtual methods. 

— Elimination of bound checks when they are provably unnecessary. 

— Elimination of unnecessary temporary objects. 



int i = 0; 

for (; (i < 12345) ;) 

{ 

int __a75 = (i * 4) ; 
int __a71 = (i * 4) ; 

float a31 = *((float *) (arrayld 96 + a71)); 

int __al84 = (i * 4) ; 

float al71 = *((float *) (arrayld 107 + al84)); 

int al63 = (i * 4) ; 

float al76 = *( (float *) (arrayld 118 + al63)); 

float __a35 = (__al71 * __al76) ; 

float a76 = ( a31 + a35) ; 

♦((float *) (arrayld + a75)) = a76; 

i = (i + 1); 

} 



Fig. 6. C Code Generated by Lunar for the Expression w=x+y+z of Figure 5 



The C code generated by Lunar for the Java expression w. assign (x. plus (- 
yexpr . times (zexpr) ) ) is shown in Figure 6. Lunar is able to eliminate all the 
virtual functions, remove all bound checks, and even get rid of the temporary 
BinaryOpExpr, Plus, and Times objects. 
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3 Benchmark Results 

We present preliminary benchmark results on a sparcvQ processor at 300 MHz, 
using gcc -03 as the back end compiler for Lunar. Figure 7 compares the 
performance of the Lunar compiler to hand-coded C for the array expression 
w = X + y * z. The JavaTran series shows the performance of Lunar on the loop 

float [] w, X, y, z; // ... 
for (int i=0; i < n; i=i+l) 
w[i] = x[i] + y[i]*z[i] ; 




Fig. 7. Benchmark results for the array expression w=x+y*z, comparing (from 
top to bottom) hand-coded C, JavaTran-style code compiled with Lunar, Java 
“Expression-templates” version compiled with Lunar (with Partial Evaluation), 
Java “Expression-templates” compiled with Lunar (no Partial Evaluation). The 
three plateaus correspond to (left to right): LI cache, L2 cache, and out-of-cache 
performance. 



The Java ET series show the performance of w. assign (x. plus (y. times (z))) 
using Array objects. When the partial evaluator is disabled (the “no PE” series). 
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Table 1. Mflops/s for the array expression w = x + y-^z, n = 1000, single 
precision. PE=Partial Evaluation. 



JVM or compiler 


“JavaTran” 


“Expression 

Templates” 


Lunar (no PE); gcc -02 backend 


33.1 


2.4 


Lunar (with PE); gcc -02 backend 


74.6 


74.6 


Snn Hotspot l.Sbeta (JIT) 


1.4 


0.4 


Transvirtual Kaffe (JIT) 


4.3 


0.7 



performance takes a drastic hit - indicating that Lunar, not gcc, is doing the 
important optimizations. 

Table 1 compares performance of Lunar to two JIT compilers (Sun Hotspot 
and Transvirtual Kaffe) . 

The partial evaluator gives a modest improvement over the JavaTran-style 
code, but the biggest improvement is for the Java version of “expression tem- 
plates” . 

Table 2. Time to compile the “expression templates in Java” example of Fig- 
ure 5, excluding C compilation time. 



Phase 


Elapsed time (ms) 


Parsing 


916 


Conversion to Lunar IL 2 


303 


Prepasses 


91 


Partial evaluation 


1569 


Unparsing to C 


678 


Total 


3557 



Table 2 shows a summary of the time taken by Lunar to compile the “expres- 
sion templates in Java” example with partial evaluation. The Lunar compiler is 
written in Java, and was run using Sun Hotspot l.Sbeta. Once the compiler is 
able to compile itself, these times should decrease. 

4 Compiler Overview 

Figure 4 shows an overview of the Lunar compiler. The front end takes Java 
code and translates it into an intermediate language, called Lunar Intermediate 
Language 2 {IL 2 ). Prior to partial evaluation, a prepass puts the code into a 
form which makes partial evaluation simpler. This code is then passed to the 
partial evaluator. The output from the partial evaluator is then translated into 
C and compiled using a C compiler. 

Table 3 summarizes the analyses and transformations currently implemented 
in Lunar. Many of these are similar to those implemented in conventional com- 
pilers. However, there are substantial differences between partial evaluation and 
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typical optimizers. Partial evaluators tend to have a flavour of symbolically ex- 
ecuting a program, whereas typical imperative compilers are more flow-graph 
based. 




Lex/Parse 
Semantic checks 
Name analysis 
Type analysis 
Translation to IL2 



Macro expansion 
Sequentiaiization 
Alpha-conversion 
Assignment elimination 
Static initializer ordering 



Liveness analysis 
Escape analysis 
Alias analysis 
Stack allocation analysis 
Copy propagation 
Constant folding 
Function specialization 
Global heap analysis 
Inlining 

Dead code elimination 
Bound check elimination 



Name mangling 
Translation from ILO 
to C code 



Low-level optimizations 
(registers, instructions, 
scheduling) 



Fig. 8. Overview of the Lunar Compiler. 



Table 3. Analyses and Transformations in the Lunar Compiler. 



Prepass transforms 


Purpose 


Macro expansion 
Sequentiaiization 
Assignment elimination 

a-conversion 

Static initializer ordering 


Expand macros used to simplify the Java front end 
Name intermediate values, simplify language structure 
Remove variable assignments; ensure that variable 
names map uniquely to values within a scope. 

Rename variables to ensure unique names 
Find initialization order for global variables; 
delete unused functions and global variables. 


Partial evaluator 


Purpose 


Liveness analysis 
Escape analysis 
Alias analysis 
Stack allocation analysis 

Copy propagation 
Constant folding 
Function specialization 

Global heap analysis 
Inlining 

Dead Code Elimination 
Bounds check elimination 


Decide how function parameters are used. 

Decide if values might escape into the heap. 

Decide if heap objects are alias-free. 

Allocate objects on the stack instead of the heap 
when possible. 

Find variables that refer to the same value. 

Fold constants, eliminate primitive operations 
Specialize functions according to known argument 
values 

Constant and copy propagation through the heap. 
Selectively inline functions to improve optimization. 
Eliminate dead variables, code, functions, and 
global variables. 

Remove bounds checks that are provably unnecessary. 
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5 Summary 

5.1 Caveats 

Lunar’s Java front end handles a modest subset of Java 1.1. It does not yet 
support threads or inner classes. It is possible that use of some Java language 
features (for example, synchronization) will make it harder to perform the opti- 
mizations described in this paper. 

Lunar emits code for a “loose numerics” version of Java. It uses the na- 
tive floating point hardware, without concern for whether this correctly imple- 
ments Java numeric semantics. This is not a requirement, but a shortcut. Strict 
numerics could be implemented, although they would likely have performance 
implications on some platforms. 

Lunar does not yet perform null pointer checking. The plan is to follow the 
example of gc j , and trap SIGSEGV signals. This does not require changing any 
of the translation or code generation; trapping is done by the hardware and 
handled in a runtime library. Hence, the benchmark results reported here are 
not compromised by our omission of null pointer checking. 

Our Java runtime does not do any garbage collection. In most of the bench- 
mark results presented, the partial evaluator eliminates all memory allocation 
inside loops. Hence the absence of a garbage collector does not affect our perfor- 
mance in a major way. The exception is the benchmarks of Table 1 which show 
Lunar results without PE. For these results, not doing garbage collection gives 
Lunar an unfair advantage over the JIT compilers. 

Some of the algorithms in Lunar are O(nlgn), and a few are quadratic in 
certain (possibly unlikely) scenarios. Lunar has not yet been tested on large 
Java programs, so it is possible we will uncover scaling problems. The plan is 
to provide a set of optimization switches -01,-02, . . ,-05 which progressively 
enable more expensive and expansive forms of partial evaluation. 

The algorithms in Lunar do not require closed-program analysis. However, 
the more of the program Lunar can see, the better it can optimize. 

Unlike G-l— I- templates. Lunar offers no guarantee of compile-time evaluation. 
There is no analogy to type template parameters in G-l — h, although Lunar can 
specialize functions based on types related by an inheritance hierarchy. 

5.2 Related Work 

There is a wealth of literature about compilers and partial evaluation, both 
separately and their intersection. The idea of resolving virtual functions through 
specialization was detailed by Dean et al [4]. Lunar achieves a similar effect 
“for free” by relying on the heap analyzer to propagate function names through 
dispatch tables. Khoo [9] used partial evaluation to compile inheritance, doing 
what would (analogously) in Java be dispatch table (or vtable) layout, but he 
did not use partial evaluation to resolve and inline virtual functions. 

Many of the G-l— I— template- like optimizations performed by Lunar are driven 
by a heap analyzer, which does constant and copy propagation through the heap. 
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This builds on a tradition of partly-static data structures in the partial evaluation 
community (e.g. [3]), and of store analyzers in the imperative world (e.g. [13,14]). 

Volanschi et al [19] describe a partial evaluator for Java which relies on user 
annotations (“specialization classes”) to guide specialization of Java programs. 
This provides finer control of specialization than Lunar, at the cost of requiring 
language extensions. 

The benefits of partial evaluation for scientific programs are well known; 
see for example [2,5,10]. Lunar is not designed to optimize scientific programs 
per se, but rather to provide reliable partial evaluation semantics which can be 
used as a driving mechanism for performing domain-specific optimizations (for 
example, “expression templates” for array classes). In this regard it has similar 
goals as two- or multi-stage languages (e.g. [15]) which seek to provide a natural 
framework for writing “program generators” . 
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